UAPI.16 File Manifest#
| Version | Changes |
|---|---|
| 0.1 | Initial Release |
This document describes a textual file manifest format that describes a collection of resources (or “blobs”). These resources may potentially be acquired over the network or stored on a disk. In particular, it can represent the properties of files arranged in a POSIX file system tree as well as partitions in a GPT partition table.
This manifest file format is inspired by the traditional UNIX
SHA256SUMS
and BSD
mtree(5)
file formats, but has various
additional features, among them:
It allows referencing remote URLs as source for acquiring file contents.
It allows extracting data “slices” from data sources, via offset and range.
Various fields of additional per-file metadata may be defined, including various fields for GPT partition table metadata.
It’s an extensible format permitting vendors and projects to add their own per-file and per-manifest fields.
It supports inline cryptographic signatures.
This format is designed with tools such as
mkosi and
systemd-sysupdate
in mind, but it’s intended to be generally useful
as a way to describe collections of files that may potentially be
acquired over the network or from local storage, and verified
cryptographically.
Terminology#
A file object (or just object for short) describes a file in the file system hierarchy or a partition in a partition table: a contiguous, sized bytestream, along with various metadata fields.
A manifest is a sequence of file, signature and trailer objects.
A root file object (or root directory object) is the top level file object of which all others are direct or indirect children, in a tree structure.
A signature object carries cryptographic signature metadata that protects all file objects listed before it.
A trailer object is the last object in the manifest and indicates termination.
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in BCP 14 RFC 2119 RFC 8174 when, and only when, they appear in all capitals, as shown here.
Manifest File Name and Media Type#
When stored in a file system directory – alongside the data files it
references – the manifest file SHOULD be named Uapi16Manifest,
possibly suffixed by a suffix indicating compression (see
below). Otherwise the recommended file suffix is .Uapi16Manifest.
This file format is identified by the media
type application/vnd.uapi.16.manifest.
General Structure#
The file is organized as an RFC 7464 JSON-SEQ sequence of file, signature and trailer objects which are described below.
Strings are sequences of octets that MUST be valid UTF-8 as defined in RFC 3629 (no overlong encodings, no surrogate code points, no NUL octets) and MUST NOT contain noncharacters (U+FDD0 to U+FDEF, and the last two code points of every plane, such as U+FFFE and U+FFFF). Strings are ordered lexicographically by octet value, with each octet treated as an unsigned integer in the range 0…255; if one string is a prefix of the other, the shorter string sorts first. For valid UTF-8 this is identical to lexicographic ordering by Unicode code point, but not to ordering by UTF-16 code units, which is what string comparison in JavaScript, Java and .NET does. No Unicode normalization, case folding, or locale-dependent collation is applied; strings that differ only in normalization form are distinct and are ordered by their octets.
All rules for strings in this specification apply to the value of a
JSON string after its escape sequences have been decoded: an escape
sequence of an unpaired surrogate such as "\uD800" makes a string
invalid, and "a\/b" is the same string as "a/b". Consumers MUST
refuse manifests with invalid strings, and MUST NOT repair them, for
example by replacing invalid sequences with U+FFFD.
The first object in the sequence always describes the enveloping
directory (or a GPT partition table header). This root file object
MUST NOT have a name field set (which identifies it as the root
file object), however, it MUST have a mediaType field set to
application/vnd.uapi.16.manifest (which identifies the sequence as a
UAPI.16 manifest).
File objects for directory entries Uapi16Manifest (or any filename
with a prefix of Uapi16Manifest.), ., .. or any path ending in
/., /.. MUST NOT exist in the sequence.
When applied to a GPT partition table the first object MAY also describe fields in the overall GPT partition table header of the disk.
Within each directory files MAY appear in any order, except as described below. However, files nested into subdirectories MUST be listed immediately following the file object of the subdirectory object. Where ordering within directories is not mandated, it is definitely RECOMMENDED to sort the entries by one of the following orderings:
By code point, i.e. by comparing the last components of their names as strings (see above), the way
strcmp()compares them.Via UAPI.10 Version Format Specification version ordering
Time-based ordering (i.e. appending new entries strictly at the end)
Sorting entries ensures the manifests become reproducible as long as the same tool and sorting algorithm is used.
A manifest sorted in the code point ordering lists its file objects in
the code point ordering of their complete names: two names are
compared component by component, starting with the first, and the
first pair of components that differ is compared as strings. If all
components of one name are also the first components of the other
one, i.e. the former is the name of a directory the latter is located
in, the former sorts first. This is the same as comparing the complete
names as strings with / taken to be lower than any other octet. Note
that comparing the complete names as strings does not result in this
ordering if names contain characters that sort before /, such as -
or .: a/x comes before a-b, since the entries of the directory
a immediately follow its file object, but a-b is the lower string.
A manifest sorted in the code point ordering SHOULD declare it, see
the order field of the root file object below.
A manifest that is generated for creating a file system or an archive
from it, or for placing its file objects in an existing one, rather
than for describing files that exist already, MUST list the entries
of every directory in the code point ordering, and MUST declare it
(see the order field of the root file object below). This does not
apply to manifests of a GPT partition table. A consumer that creates
a file system or an archive from a manifest, or places its file
objects in an existing one, MAY refuse a manifest that is not in the
code point ordering, rather than sort its file objects. Such a
consumer MAY also refuse a manifest that does not declare the
code point ordering, even if its file objects are in it.
If file objects are placed in an existing file system or archive whose manifest declares an ordering, and they are added to that manifest, it MUST preserve the ordering and keep declaring it.
For every file object other than the root file object, the manifest
MUST list the file object of the directory it is located in before it:
the file object of type dir whose name is its name without the last
component, or the root file object if its name has a single
component. Together with the requirement that files nested into
subdirectories immediately follow the file object of the subdirectory
(see above), this means that the file objects of a manifest are listed
in tree order: every directory comes before the file objects below it,
and these immediately follow it.
The name field of file objects MUST be uniquely assigned within a
manifest file.
A consumer that finds a name listed twice, or file objects below a directory that do not immediately follow the file object of that directory, SHOULD refuse the manifest rather than pick one of the conflicting file objects. Otherwise different consumers end up disagreeing on what the same manifest describes. Both can be checked while reading: misplaced file objects with a stack of the directories that are still open, since every directory is listed, and repeated names by comparing each name with the previous one if the manifest declares the code point ordering.
If the manifest encodes a GPT partition table, the name field MUST
still be assigned for every file object, even if gptLabel is defined
too. The former MUST still be unique within the manifest; the latter
does not have to be.
For manifests covering a directory tree with nested directories – as
opposed to a flat directory listing without file objects of type
dir –, subdirectory listings MAY either be specified inline
(typically preferable) or MAY be imported via a separate
“sub-manifest” file covering just this subdirectory (and potential
sub-subdirectories, …), by referencing the separate UAPI.16 manifest
file in the contents field of the file object.
When sub-manifests are used, the directory they cover is described
twice: by the dir file object in the outer (enveloping) manifest,
and by the root file object of the inner sub-manifest. The inode
attributes of this directory (i.e. fields such as mode, uid,
gid, userName, groupName, mTimeNSec, extendedAttributes, but
not fields referring to the sub-manifest itself, such as name,
size, contents or sha256) SHOULD be listed in the outer
manifest, and MAY also be listed in the inner manifest. If listed in
both, they MUST NOT contradict each other. It is RECOMMENDED that the
inode attributes declared in the outer manifest are a superset of
those declared in the inner manifest.
Zero, one or more signature objects MAY be placed in the manifest
after all file objects. They provide cryptographic protection for all
file objects coming before. Signature objects are recognized by their
mediaType field set to application/vnd.uapi.16.signature.
Finally, a single trailer object MUST be placed in the manifest, as
the last object in the sequence. Trailer objects are recognized by their
mediaType field set to application/vnd.uapi.16.trailer. Each
sequence has exactly one trailer object. Any data coming after the
trailer object is invalid, and MUST result in a parse failure.
┌─────────────────────────────────────────────────────────┐
│ Root file object (application/vnd.uapi.16.manifest) │
├─────────────────────────────────────────────────────────┤
│ File object #1 │
│ File object #2 │
│ File object #3 │
│ File object … │
├─────────────────────────────────────────────────────────┤
│ Signature object #1 (application/vnd.uapi.16.signature) │
│ Signature object #2 (application/vnd.uapi.16.signature) │
│ Signature object #3 (application/vnd.uapi.16.signature) │
│ Signature object … │
├─────────────────────────────────────────────────────────┤
│ Trailer object (application/vnd.uapi.16.trailer) │
└─────────────────────────────────────────────────────────┘A manifest MUST consist of at least the root file object and the trailer object, the file and signature objects are optional.
A record, i.e. the bytes from its record separator up to the next record separator, MUST NOT be larger than 4 MiB. Consumers MUST accept records within this limit, and MAY refuse larger ones.
Sizes, offsets and timestamps shall use non-negative integer
JSON numbers, written as
decimal digits without sign, fraction or exponent, and MAY use values
above 2⁵³. Note that mTimeNSec is above 2⁵³ for every point in time
after mid-April 1970, hence JSON implementations that store numbers as
64-bit floating point values cannot process the timestamps of a
manifest losslessly. Consumers MUST process these numbers as unsigned
64-bit integers.
File Object#
This is an object that encodes information about an individual file, or about the root directory of the manifest.
{
"name" : "…",
"mediaType" : "application/vnd.uapi.16.manifest",
"order" : "codepoint",
"features" : [ "…", … ],
"type" : "reg",
"size" : 33,
"major" : 4711,
"minor" : 815,
"mode" : 123,
"uid" : 1000,
"userName" : "lennart",
"gid" : 1000,
"groupName" : "lennart",
"mTimeNSec" : 1777615169123123123,
"inodeToken" : 4711,
"extendedAttributes" : { "user" : { "…" : { "text" : "…" }, "…" : { "base64" : "…" } } },
"gptLabel" : "…",
"gptUuid" : "…",
"gptTypeUuid" : "…",
"gptDiskUuid" : "…",
"gptFlags" : 4711,
"gptFlagNoAuto" : true,
"gptFlagGrowFileSystem" : true,
"readOnly" : true,
"flagHidden" : true,
"flagSystem" : true,
"flagArchive" : true,
"fatSerial" : "1234-ABCD",
"fsUUID" : "…",
"fsLabel" : "…",
"flagAppendOnly" : true,
"flagImmutable" : true,
"flagNoAtime" : true,
"flagNoCow" : true,
"flagNoDump" : true,
"contents" : [{…}, …],
"sha256" : "…",
"validSinceUSec" : 4711,
"validUntilUSec" : 4711,
"validity" : "revoked"
}Fields#
All fields are generally optional (but might be REQUIRED for some
objects under certain conditions, see below). If a field shall not be
set it can either be set to null or simply be omitted in the JSON
object (RECOMMENDED), both ways MUST be considered equivalent.
Unnecessary whitespace/line breaks SHOULD be omitted when formatting the JSON records. Fields within JSON objects SHOULD be sorted by their field names in the ordering of strings (see above).
A JSON object in a manifest MUST NOT contain the same key twice, and consumers MUST refuse manifests that contain such objects.
Two file objects are equivalent if they carry the same fields with the
same JSON values. This does not depend on the order of the fields in
JSON objects or on how values are written in the JSON text: strings
are compared after their escape sequences have been decoded (see
above), numbers by their value, arrays element by element and JSON
objects field by field. A field set to null counts as not set, also
in nested JSON objects. Since values are not interpreted any further,
this also applies to fields a tool does not know. Hence hexadecimal
digests in uppercase and in lowercase, or the same data in the two
Base64 alphabets, are not equivalent. The values of inodeToken are
specific to their manifest, so they are not compared between file
objects of different manifests. Tools that compare or combine
manifests SHOULD compare file objects for equivalence rather than by
the bytes of their JSON encoding.
Generators SHOULD NOT set fields to the default this specification
defines for them, for example type set to reg on a file object
other than the root file object, validSinceUSec set to 0, validity
set to good, readOnly set to false or offset of a contents
object set to 0, so that file objects describing the same file
compare as equivalent.
If not otherwise indicated, fields are of type string.
The mediaType field is REQUIRED on the first file object in the
sequence (i.e. the top-level root directory object), and MUST have the
value "application/vnd.uapi.16.manifest". It MAY also be set on
other file objects in the sequence (to the same value), but this is
NOT RECOMMENDED.
The order field SHOULD be set on the root file object of a manifest
sorted in the code point ordering, and MUST NOT be set on any other
file object. It MUST be set on a manifest generated for creating a
file system or an archive, or for placing its file objects in an
existing one, as described above. If set to codepoint, the names of
the file objects strictly increase in the code point ordering of
complete names (see above). Hence the file objects of every directory
are listed in strictly increasing code point ordering, and no two file
objects have the same name. A consumer that processes a manifest in
reliance on this MUST check it while reading, and MUST refuse the
manifest if it does not hold. No other values are defined at this
time, consumers MUST treat other values as if the field was not set.
This field describes the manifest rather than the root directory:
tools combining manifests MUST NOT compare it between their inputs,
and MUST set it on their output only if it holds for the output.
The features field (array of strings) MAY be set on the root file
object, and MUST NOT be set on any other file object. It lists the
file system and partition table features the manifest records, and
thus which fields its file objects can carry. If not set, it defaults to device, gid,
hardlink, mode, sha256, size, symlink and uid. A file
object MUST NOT carry a field that belongs to a feature the manifest
does not list. The following values are defined:
| Value | Fields |
|---|---|
mode | mode |
uid | uid |
gid | gid |
user-name | userName |
group-name | groupName |
size | size |
sha256 | sha256 |
symlink | contents of file objects of type lnk |
device | major and minor |
hardlink | inodeToken |
mtime | mTimeNSec |
user-xattr | the user key of extendedAttributes |
flag-hidden | flagHidden |
flag-system | flagSystem |
flag-archive | flagArchive |
flag-append-only | flagAppendOnly |
flag-immutable | flagImmutable |
flag-no-atime | flagNoAtime |
flag-no-cow | flagNoCow |
flag-no-dump | flagNoDump |
read-only | readOnly |
fs-uuid | fsUUID |
fs-label | fsLabel |
fat-serial | fatSerial |
gpt-label | gptLabel |
gpt-uuid | gptUuid |
gpt-type-uuid | gptTypeUuid |
gpt-disk-uuid | gptDiskUuid |
gpt-flags | gptFlags |
gpt-flag-no-auto | gptFlagNoAuto |
gpt-flag-grow-file-system | gptFlagGrowFileSystem |
Fields that do not belong to any of these features, such as type,
contents of file objects of other types, validity,
validSinceUSec and validUntilUSec, are not affected. If symlink
is listed, every file object of type lnk carries its target as
literal in contents. For hardlink, user-xattr, fs-label,
gpt-flags, read-only, and the flag-… and gpt-flag-… values,
the feature also defines what a missing field means. If the feature is
listed, a file object that does not carry the field is not subject to
hardlinks, has no extended attributes in the user namespace, has no
file system label, has all partition attribute flags cleared
(i.e. gptFlags is 0), or has the flag cleared (e.g. readOnly is
false), respectively. If it is not listed, nothing is known about
these properties.
The order of the values carries no meaning. A value MUST NOT be listed
twice, and the values SHOULD be sorted in the ordering of strings (see
above). Consumers MUST ignore values they do not know. A consumer that
processes a manifest in reliance on its features MUST check them while
reading, and MUST refuse the manifest if they do not hold. Like
order, this field describes the manifest rather than the root
directory. Tools that compare or combine manifests MUST compare their
features, as sets, before they compare file objects. If the features
differ, such a tool MUST either refuse the manifests, or first remove
the fields of the features that not all of them list from their file
objects. A tool that combines manifests sets features on its output
accordingly.
If name is not specified the file object stores information about
the top-level root file object. This file object always comes first
in the file object sequence. Or in other words, the root file object
(the first object in the sequence) has mediaType but not name
set, while all other file objects have name set.
The name MUST be a valid
UTF-8 POSIX path (i.e. using the slash /
as component separator), MUST NOT contain control characters
(ASCII 0…31, 127) and MUST NOT contain “.”, “..” or empty strings as components
of the path. It MUST NOT start or end with a slash. It MUST NOT have
multiple slashes in immediate sequence (or in other words, the path
must be normalized and relative). The name MUST NOT be identical to
“Uapi16Manifest” or be prefixed with “Uapi16Manifest.” (it is
however acceptable if these filenames appear as components of paths
further down the tree). Files whose paths do not follow these rules
cannot be encoded in manifest files defined by this
specification. The name MUST be uniquely assigned within a
manifest. It MUST NOT be longer than 4095 bytes. The components of
the path are not limited on their own.
The type field encodes the inode type of the file object. It takes
one of reg, dir, lnk, fifo, chr, blk, sock (these values
follow low-level UNIX/Linux naming of these inode types). Other values
MUST NOT be used. If not specified it defaults to dir for the first
object in the sequence (i.e. the root file object) and reg for all
others. Note that when using the manifest format to list partition
contents each partition MUST be listed as a file of type reg.
The size (unsigned integer) field MAY be used in file objects of
type reg, lnk, dir and encodes the size of the contents of the
file, in bytes. It MUST NOT be used on file objects of other
types. Specifically, for regular files this is the size as reported by
the file system. In case of symlinks: this is the length of the
symlink target string. In case of dir this is the size of a UAPI.16
file manifest generated for the subdirectory (which only applies if
the directory contents are not provided inline, but via a reference
to a separate sub-manifest file, see above).
The major (unsigned integer) field MAY be used in file objects of
type blk and chr, it encodes the major device number of the device
node. It MUST NOT be used on objects of other types. Similarly,
minor (same type and condition) encodes the minor device number.
The mode (unsigned integer) field encodes the UNIX access mode of
the file object. It MAY be used in file objects of all inode types,
except lnk, where it MUST NOT be used. Note that while UNIX access
modes are typically written in octal, this one is encoded in a regular
JSON number, i.e. decimal. The valid range is 0…4095 (i.e. 0o0000 to
0o7777). Values outside this range MUST NOT be used.
The uid (unsigned integer) field is the numeric owning UNIX user ID of the
file. Similarly, gid (same type) is the numeric owning UNIX group ID
of the file. Both fields – if set – MUST be unsigned 32-bit integers,
i.e. in the range 0…4294967295 (i.e. 0…2³²-1). However, the values
65535 (i.e. 2¹⁶-1, which is -1 as unsigned 16-bit integer, as used by
the legacy 16-bit UID/GID system calls) and 4294967295 (i.e. 2³²-1,
which is (uid_t) -1/(gid_t) -1, i.e. -1 as unsigned 32-bit
integer) MUST NOT be used, as these are commonly used as special
“invalid” or “unset” markers.
The userName field is the owning UNIX user name of the file;
similarly, groupName is the owning UNIX group name of the file. Both
fields SHOULD contain a valid UNIX user name or group name,
respectively. (Note that UNIX user/group
name validity is not very well defined, hence it’s recommended to
liberally accept strings here, maybe with the exception of empty
strings, fully numeric strings, and those containing control
characters).
The mTimeNSec field (unsigned integer) is the UNIX file modification
timestamp of the file object, in nanoseconds since the UNIX epoch (Jan
1st, 1970).
The inodeToken field (unsigned integer) shall be a unique numeric
identifier for any file that SHALL be hardlinked within the manifest
tree. If unspecified or 0 the file is not subject to hardlinks (if the
manifest lists the hardlink feature, see above), otherwise any pair
of file objects that shall be hardlinks to the same inode SHALL carry
the same inodeToken value. In order to ensure
reproducibility inodeToken SHALL be assigned as a counter starting
at 1 for the first file that shall be hardlinked, and increasing by 1
for each subsequent file that shall be hardlinked but has not been
hardlinked in the manifest so far. When sub-manifests are used,
hardlinking between the upper and lower manifest is not supported –
the inodeToken identifiers are specific to the manifest they are
specified in. This field MUST NOT be used for file objects of type
dir.
A non-zero inodeToken MAY be carried by a single file object only,
for example if the other names of the file are outside of the tree
the manifest covers, and consumers MUST accept that. All file objects
sharing the same non-zero inodeToken describe the same inode: they
MUST be of the same type, and MUST carry identical mode, uid,
gid, userName, groupName, mTimeNSec, size, sha256,
major and minor fields, in addition to the extendedAttributes
and flag fields described below.
The extendedAttributes field (object) encodes extended attributes
of the file object, i.e. additional name/value pairs attached to the
inode (see for example Linux'
xattr(7)). It
is an object whose keys are extended attribute namespace identifiers
(without any separator character). Each value is another object (the
namespace object), whose keys are the attribute names within that
namespace, without the namespace prefix (e.g. on Linux, foo for the
extended attribute user.foo). Each value of a namespace object is
an extended attribute value object, described below.
Currently only the user extended attribute namespace is supported by this
specification. It encodes unprivileged, freely definable extended attributes,
such as those in the user. namespace on Linux. Additional namespaces (for
example covering security labels or other privileged attributes, such as the
security. and trusted. namespaces on Linux) may be defined in future
revisions of this specification. Namespace identifiers not defined by this
specification MUST NOT be used. When parsing, unrecognized namespace
identifiers MUST be ignored (but SHOULD probably be logged about). Note that
some operating systems expose other file metadata via the extended attribute
interface (for example POSIX ACLs via the system.posix_acl_access and
system.posix_acl_default extended attributes on Linux). Such metadata is
expected to be encoded in separate, dedicated fields, not via this field.
Attribute names MUST be valid UTF-8, MUST NOT be empty and MUST NOT contain control characters (ASCII 0…31, 127). Extended attributes whose names do not follow these rules cannot be encoded in manifest files defined by this specification.
If a namespace object is present (and not null) it MUST list all
extended attributes of that namespace of the file object, i.e. an
empty namespace object indicates that the file object has no extended
attributes in that namespace. If a namespace key is absent (or set to
null) no information about the extended attributes of that
namespace is conveyed, unless the manifest lists the feature for that
namespace (user-xattr for user, see above): then the file object
has no extended attributes in that namespace.
All file objects sharing the same non-zero inodeToken MUST carry
identical extendedAttributes fields, as extended attributes are a
property of the inode, not of the directory entry.
Extended attributes are not covered by the sha256 field (which only
covers the file contents), but are protected by the manifest signature
like all other fields.
When file objects are placed in a GPT partition table, this field has no effect and SHOULD be ignored.
Note that file systems and operating systems typically impose limits on extended attributes, for example on the length of names and values, their total size per inode, or the inode types they may be attached to. This specification does not mandate any such limits. Consumers applying a manifest to a file system SHOULD handle such restrictions gracefully.
The gpt* fields encode fields that are needed when placing these
resources in a GPT partition table entry. They only apply to file
objects of inode type reg, with the exception of gptDiskUuid which
only applies to the root file object (which has an inode type of
dir). These fields MUST NOT be used on file objects of other types.
The gptFlags field (unsigned integer) encodes the raw 64-bit
partition attribute flags of the GPT partition table entry, as defined
by the UEFI
specification.
The gptFlag* fields are of boolean type. They provide access to
individual partition attribute flags by name: gptFlagNoAuto and
gptFlagGrowFileSystem correspond to the flags of the same name
defined in the Discoverable Partitions
Specification (bits 63 and
59, respectively). The read-only flag (bit 60) is accessed via the
generic readOnly field instead, see below. If both gptFlags and a
gptFlag* field or readOnly are set for the same file object, they
MUST be consistent. For this, a field that is not set counts as set to
false (or 0 in case of gptFlags) if the manifest lists its feature
(i.e. gpt-flags, gpt-flag-no-auto, gpt-flag-grow-file-system or
read-only, see above).
The gpt*Uuid fields take a UUID, and MUST be formatted as per RFC
9562 Section
4. It SHOULD be
formatted in lowercase. gptUuid stores the partition UUID for the
entry, and gptTypeUuid the partition type UUID.
The gptLabel field MUST NOT be longer than 36
UTF-16 code units (72
bytes), as defined by the UEFI specification. If gptLabel is
unspecified, and the data is written to a GPT partition it SHOULD use
the value of name as label instead. Note that this might require
mangling, as GPT partition labels have smaller size limits than file
names. Also note that unlike the name this field does not have to be
uniquely assigned among all entries in the same manifest.
The gptDiskUuid field MUST be a UUID in RFC 9562 Section 4
formatting. It SHOULD be formatted in lowercase. It encodes the disk
UUID for the GPT partition table. In contrast to the other gpt*
fields this one MUST NOT be set on any object except for the root
directory object (i.e. the first file object in the sequence), as it
encodes a field of the GPT partition table header rather than the GPT
partition entry header.
readOnly (a boolean) is a generic read-only marker for the file
object, that is mapped onto the read-only concept of whatever storage
the file object is placed on. Specifically, it is relevant for
initializing the read-only flag (bit 60, as defined in the
Discoverable Partitions Specification) of GPT partition table entries,
as well as the DOS ATTR_READ_ONLY attribute (0x01), as stored for
example in FAT directory entries (see below). It should also be
reflected in the POSIX ‘w’ access mode bit when storing the data in a
regular file on disk. Depending on context, it hence corresponds to
the UNIX access mode, the DOS ATTR_READ_ONLY attribute or the GPT
read-only partition attribute flag, but it MAY be set independently of
the mode and gptFlags fields. A file object MUST NOT carry this
field unless the manifest lists the read-only feature (see
above). If unspecified, and the feature is listed, the file object is
not read-only. If unspecified, and the feature is not listed, nothing
is known about the property, and it defaults to false when the file
object is placed somewhere (but see below). If both mode and
readOnly are set for the same file object, they MUST NOT contradict
each other: if any of the three ‘w’ bits in mode is set, readOnly
MUST be false, otherwise it MUST be true. Similarly, if both
gptFlags and readOnly are set, they MUST NOT contradict each other
(see above).
The flagHidden, flagSystem and flagArchive fields encode DOS
file attributes, as implemented by
FAT
(such as the
UEFI
EFI System Partition),
exFAT,
NTFS and other file systems with DOS attributes. The fatSerial field
encodes the volume serial number of FAT and exFAT file systems. If the
file objects are placed in a file system that does not implement the
respective concept, these fields have no effect and SHOULD be ignored.
The flagHidden, flagSystem and flagArchive fields are of
boolean type, and control the DOS attribute flags of the file object
(which use the same bit values on all file systems listed above):
flagHidden corresponds to the ATTR_HIDDEN
attribute (0x02), flagSystem to ATTR_SYSTEM (0x04), and
flagArchive to ATTR_ARCHIVE (0x20). (There is no separate field
for the ATTR_READ_ONLY attribute (0x01): the generic readOnly
field described above is used for that.) (On Linux these attributes
may be queried and set on FAT file systems via the FAT_IOCTL_GET_ATTRIBUTES and
FAT_IOCTL_SET_ATTRIBUTES ioctls.) These fields only apply to file
objects of inode type reg and dir, and MUST NOT be used on file
objects of other types. They MUST NOT be used on the root file object
either, as the root directory of FAT and exFAT file systems has no
directory entry, and hence no attributes. If any of these fields is not set, the
state of the corresponding flag is not defined, unless the manifest
lists the feature for the flag (see above): then the flag is cleared.
When placing a file object in a file system with DOS attributes, the
ATTR_READ_ONLY attribute SHOULD be set if readOnly is true, and
SHOULD NOT be set if it is false. If readOnly is not set, and the
manifest does not list the read-only feature, but mode is set, an implementation MAY derive the attribute from the
latter instead of applying the default of readOnly (i.e. set it if
none of the three ‘w’ bits in mode is set).
The fatSerial field encodes the 32-bit volume serial number (also
known as “volume ID”) of the FAT or exFAT file system. It MUST be formatted the
way FAT volume serial numbers are usually displayed: as two groups of
four hexadecimal digits, separated by a hyphen (e.g. 1234-ABCD),
where the first group encodes the upper 16 bits and the second group
the lower 16 bits of the serial number. Parsers MUST parse this
case-insensitively, but it SHALL be formatted in uppercase. Other
formats (e.g. without the hyphen, or with fewer digits) MUST NOT be
used. If unspecified, the implementation SHOULD pick a suitable serial
number. For reproducibility it is RECOMMENDED to derive it
deterministically from data in the manifest rather than randomly or
from the current time. The precise derivation is
implementation-defined, hence manifest authors who require identical
serial numbers across implementations SHOULD set this field
explicitly.
The fatSerial field MUST NOT be set on any object except for the
root directory object (i.e. the first file object in the sequence), as
it encodes a property of the file system as a whole, rather than of
an individual file.
The fsUUID and fsLabel fields encode the identity of the file
system the file objects are placed in, i.e. its UUID and its label, as
reported on Linux by the FS_IOC_GETFSUUID and FS_IOC_GETFSLABEL
ioctls, respectively (and as typically shown by tools such as
blkid(8)). They
are applied when a new file system is created and populated from the
manifest. When file objects are placed in an existing file system
instead, implementations SHOULD check that the UUID and label of that
file system match the values specified in these fields (if set), and
SHOULD fail the operation if they do not. When file objects are placed
in a GPT partition table these fields have no effect. These fields MUST
NOT be set on any object except for the root directory object
(i.e. the first file object in the sequence), as they encode
properties of the file system as a whole. When the manifest is used as
a sub-manifest (i.e. its root directory object does not correspond to
the root directory of a file system) these fields SHOULD be ignored.
The fsUUID field MUST be a UUID in RFC 9562 Section 4 formatting. It
SHOULD be formatted in lowercase. Note that some file systems
distinguish between the user-visible file system UUID and a separate
internal UUID used for on-disk metadata (e.g. the ext4
metadata_csum_seed feature, or the btrfs and XFS metadata UUIDs):
this field refers to the former, i.e. the UUID reported by
FS_IOC_GETFSUUID. If unspecified, the implementation SHOULD pick a
suitable UUID. For reproducibility it is RECOMMENDED to derive it
deterministically from data in the manifest rather than randomly or
from the current time. The precise derivation is
implementation-defined, hence manifest authors who require identical
UUIDs across implementations SHOULD set this field explicitly.
FAT and exFAT file systems do not have a UUID, but only a 32-bit volume
serial number, see fatSerial above. When placing file objects in
such a file system, and fsUUID is set but fatSerial is not,
implementations SHOULD derive the serial number from the first four
bytes of the UUID, i.e. from the first eight hexadecimal digits of its
string representation (e.g. fsUUID set to
1234abcd-5678-4ef0-9abc-def012345678 results in a serial number of
1234-ABCD). If both are set, fatSerial takes precedence.
The fsLabel field MUST be valid UTF-8 and MUST
NOT contain control characters. Different file systems impose different
restrictions on the length of the label and on the set of characters
permitted in it (e.g. labels may be at most 11 characters on FAT, 12
bytes on XFS, and 16 bytes on ext4). If the label cannot be represented
on the target file system, the implementation SHOULD fail the
operation. If unspecified, no label is set (or the implementation’s
default is used, unless the manifest lists the fs-label feature, see
above).
The flagAppendOnly, flagImmutable, flagNoAtime, flagNoCow and
flagNoDump fields are of boolean type, and control a subset of
the Linux inode flags, as shown and changed by
lsattr(1) and
chattr(1),
and accessible via the FS_IOC_GETFLAGS/FS_IOC_SETFLAGS ioctls (see
ioctl_iflags(2)):
| Field | Linux Flag |
|---|---|
flagAppendOnly | FS_APPEND_FL |
flagImmutable | FS_IMMUTABLE_FL |
flagNoAtime | FS_NOATIME_FL |
flagNoCow | FS_NOCOW_FL |
flagNoDump | FS_NODUMP_FL |
These fields only apply to file objects of inode type reg and dir
(including the root file object), and MUST NOT be used on file objects
of other types. If any of these fields is not set, the state of the
corresponding flag is not defined, unless the manifest lists the
feature for the flag (see above): then the flag is cleared.
Only flags that are implemented by commonly used Linux file systems,
and that may be changed directly by userspace, are covered by this
specification. Flags that merely reflect internal state of the inode
(such as FS_EXTENT_FL, FS_ENCRYPT_FL, FS_VERITY_FL or
FS_INLINE_DATA_FL) are out of scope. Note that not all file systems
implement all flags (e.g. FS_NOCOW_FL is only implemented by
copy-on-write file systems such as btrfs). If a flag is requested that
the target file system does not support (including the case where the
file system does not support Linux inode flags at all), an
implementation SHOULD ignore it (and MAY log about this). An
implementation MAY fail the operation instead if the caller explicitly
requested strict behavior. Some of
these flags (in particular flagAppendOnly and
flagImmutable) require privileges to be set
(CAP_LINUX_IMMUTABLE).
When applying a manifest to a file system, implementations SHOULD set
flagAppendOnly and flagImmutable only after the file
contents and all other metadata of the file object have been written
(and in case of directories: after all entries within it have been
created), as these flags prohibit later modifications. Conversely,
flagNoCow SHOULD be set on regular files while they are still
empty, i.e. before the file contents are written, as file systems
typically do not permit changing it (or do not apply it) on non-empty
files. When set on directories, file systems typically propagate some
of these flags (in particular FS_NOCOW_FL) to files subsequently
created within them.
All file objects sharing the same non-zero inodeToken MUST carry
identical values for these fields, as these flags are a property of the
inode, not of the directory entry.
When file objects are placed in a GPT partition table these fields have no effect and SHOULD be ignored.
contents is an array of contents objects, as described below. This
field only applies to file objects of type reg, lnk, dir. It
MUST NOT be used on file objects of other types. In case of regular
files this references or contains the file contents. In case of
symlinks this references or contains the symlink target string. In
case of directories this references or contains the sub-manifest for
the directory (if applicable, see above). If this field is not set and
the file object is of type reg, an array with a single contents
object SHALL be implied, that uses file to reference a file of the
name specified in name (without encoding or offset). Or in other
words: if no contents is specified in a file object of a
Uapi16Manifest file, then it should be looked for in a file of the
name name in the same directory.
sha256 is the
SHA256
hash of the file contents. It MUST be formatted
in 64 hexadecimal characters. Parsers MUST parse this
case-insensitively, but it SHALL be formatted in lowercase. This field
only applies to file objects of type reg, lnk, dir. It MUST NOT
be used on file objects of other types. The hash value covers the
contents referenced or contained in the contents field described
above. Additional hash algorithms may be defined in future, in which
case separate fields will be defined for each. If the size field is
zero, this field MUST either be unset (RECOMMENDED) or be identical to
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855,
i.e. the hash of an empty string.
valid*USec is in µs since UNIX epoch UTC, and shall be an unsigned
integer. It defines a validity time window of the resource. The file
object SHOULD NOT be accessed or consumed outside of the specified
time window. If validSinceUSec is not specified it MUST default to
0, if validUntilUSec is not specified the object never expires. If
a directory file object is invalidated via these two settings, then
all file objects within it are invalidated implicitly too. This
specifically means that the time window specified on the root
directory object (i.e. the first one in the sequence) may invalidate
the whole manifest file.
validity takes a string, and implements an enumeration: one of
good or revoked. Other values MUST NOT be used at this time (if
encountered while parsing any unrecognized strings MUST be considered
equivalent to good, in order to cater for future extensions of this
field). If unspecified, defaults to good. If set to revoked the
file SHOULD NOT be considered anymore for automatic consumption by the
client, and if a client already acquired it before it SHOULD consider
removing it again or otherwise ensure it’s not used anymore.
Extended Attribute Value Object#
{
"text" : "…",
"base64" : "…"
}This object encodes the value of an individual extended attribute, see
the extendedAttributes field of the file object above.
Fields#
text and base64 are mutually exclusive: exactly one of them MUST
be set. text contains the attribute value as JSON string; the value
is its UTF-8 encoding, without any trailing NUL byte. text MUST NOT
contain the NUL character (\u0000). base64 contains the attribute
value in Base64 encoding (either alphabet, as for the literal field
of contents objects).
Generators SHOULD use text if – and only if – the attribute value is
valid UTF-8 and contains no noncharacters and no NUL bytes (this
includes trailing NUL bytes, which some programs append to textual
values), and base64 otherwise. Following this rule strictly ensures
reproducible output.
Empty attribute values are valid, and SHOULD be encoded as
{ "text" : "" }.
Out of Scope#
Note that the above does not support – on purpose – the following common UNIX file or GPT partition table properties:
- Access timestamps, birth timestamps, change timestamps
- Inode numbers and other non-portable file IDs (such as
name_to_handle_at()style FID) - Backing device inode major/minor
- Hardlink link counts
This is because all of these will quite likely differ if the same manifest is applied to different file systems.
Contents Object#
{
"file" : "…",
"url" : "https://…",
"httpHeaders" : { "Authorization" : "Bearer …", "…" : "…" },
"literal" : "TmV2ZXIgZ29ubmEgZ2l2ZSB5b3UgdXAsIG5ldmVyIGdvbm5hIGxldCB5b3UgZG93bg==",
"encoding" : "gzip",
"encodedSize" : 2011,
"encodedSha256" : "0ef149998289474e4bb31813edda6ad7f3c991b2d8dec6e8fe4db7a1f039f2d1",
"originalSize" : 4711,
"originalSha256" : "afe2882a75c0c603b8b8de5efc20b9043fa33eb0e0cc0d91e54e1dfc2f5fdb8e",
"offset" : 22
}Contents objects indicate where the data of a reg, lnk, dir type
file object may be found. The contents field of the file objects (see
above) contains an array of objects of this type.
Fields#
file, url and literal are mutually exclusive: at most one of them
MAY be set. file references
a file in the same location as the manifest file itself was found,
url references a full URL, and literal contains inline
Base64 data. url MUST NOT use any other schemes than http://… and
https://…. It is RECOMMENDED to use literal to encode the target
of symbolic link file objects. If none of these fields are set,
behavior equivalent to file being set to the file object’s name
field SHOULD be implied.
httpHeaders (object) encodes additional
HTTP header fields to include
in the HTTP GET (or HEAD) requests issued when acquiring the
resource referenced by url. It MUST NOT be set unless url is set
too. Each key of the object is an HTTP field name, each value a JSON
string containing the field value. Field names MUST be valid HTTP
field names as per RFC 9110 Section
5.1 (i.e. MUST
consist of tchar characters only, and MUST NOT be empty), and SHOULD
be formatted in their canonical capitalization (e.g. Authorization,
X-Foo-Bar). Since HTTP field names are case-insensitive, the keys
of the object MUST be unique when compared case-insensitively. Field
values MUST be valid HTTP field values as per RFC 9110 Section
5.5, and in
particular MUST NOT contain CR, LF or NUL characters, and SHOULD NOT
have leading or trailing whitespace. If the same field shall be sent
with multiple values, they SHOULD be combined into a single,
comma-separated field value, as per RFC 9110 Section
5.3.
Header fields that are typically owned by the HTTP client
implementation itself – for example because they relate to connection
management, transfer encoding, or the range and encoding logic
described in this specification – SHOULD NOT be set via
httpHeaders. Examples of such fields are Host, Range and
Accept-Encoding. Consumers MAY refuse or ignore any fields they do
not wish to send (including, but not limited to, fields they consider
owned by the HTTP client), in which case they SHOULD consider the
contents object unsuitable and proceed with the next one in the
contents array.
The header fields SHALL only be sent in requests to the origin (as per
RFC 6454) of the URL
specified in url. If an HTTP redirect to the same origin is
encountered, the client MAY copy the header fields to the redirected
request, but SHOULD only do so for fields whose semantics it
understands and for which it hence knows that copying them is not
problematic.1 If an HTTP redirect points to a different
origin, the header fields MUST NOT be sent to it (as they might carry
credentials or other sensitive data). Note that the httpHeaders
field is part of the manifest and hence protected by its signature,
but not otherwise kept confidential: credentials SHOULD only be placed
in it if the manifest itself is not publicly accessible.
literal may be encoded either in regular Base64 (RFC 4648 Section
4) or in URL-safe
Base64 (RFC 4648 Section
5). Consumers MUST
be able to deal with either alphabet.
encoding specifies the encoding of the specified resource. It
accepts the same encoding specifiers as HTTP’s
Content-Encoding:
field, e.g. gzip or
zstd. Exactly one encoding
specifier MUST be specified (i.e. unlike HTTP, a comma-separated list
of multiple stacked encodings is not permitted). The specifier MUST be
a name registered in the IANA HTTP Content Coding
Registry,
in lowercase. Aliases (such as x-gzip) MUST NOT be used. If not set
(i.e. omitted or null), no encoding is used. Note that – unlike HTTP – the special
encoding specifier
identity MUST
NOT be used to indicate the absence of an encoding: this field MUST be
left unset instead. Consumers SHOULD refuse contents objects with
encoding set to identity.
originalSize is the size of the full, decoded data, encodedSize of
the full, encoded data. If encoding is not specified, encodedSize
SHOULD NOT be specified. (If specified anyway, it MUST match
originalSize). Both fields are unsigned integer type.
originalSha256 is the SHA256 sum of the full, decoded data,
encodedSha256 of the full, encoded data. If encoding is not
specified, encodedSha256 SHOULD NOT be specified (if specified
anyway it MUST match originalSha256). Both are 64 character
hexadecimal strings (case-insensitive, but lowercase RECOMMENDED). If
encodedSize is zero then encodedSha256 MUST be unset (RECOMMENDED)
or be identical to
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855,
i.e. the hash of an empty string. Similarly, if originalSize is zero,
originalSha256 MUST be unset (RECOMMENDED) or be the hash of an
empty string.
If offset is specified, then the contents of the file SHALL be
acquired from the specified offset in the decoded data. If the field
is not specified it defaults to 0. The data to extract begins at the
specified offset, and extends over the size specified in the size
field of the file object. Note that this means that the sum of
offset and size MUST be less than or equal to originalSize (if
all are specified), so that the slice does not extend beyond the
source data.
Signature Objects#
Optionally, manifest files MAY include one or more signature objects. If included these objects MUST appear in the JSON-SEQ sequence after the last file object. Each such object MUST follow the following general structure:
{
"mediaType" : "application/vnd.uapi.16.signature",
"mechanism" : "openpgp",
"key" : "…",
"data" : "…"
}Fields#
The mediaType is set to application/vnd.uapi.16.signature for
objects of this type. This field MUST be used to recognize the
signature object, and distinguish it from a file object.
The mechanism field MUST be set to a string identifying the
signature mechanism. Currently defined are openpgp for
OpenPGP compatible
signatures, and pkcs7 for PKCS
#7. Besides mediaType
this is the only required field, the other fields’ existence depends
on the used mechanism.
The key field MAY point to a key identifier, as appropriate for the
selected mechanism. For the openpgp mechanism this is the key
fingerprint in hexadecimal characters (lowercase). For pkcs7 this is
the SHA256 hash of the signing key’s
X.509 certificate in
DER form,
formatted in lowercase hexadecimal characters. The key field may be
used to optimize searching for the right key to authenticate with, but
its inclusion is OPTIONAL.
The data field contains the signature data in Base64 encoding
(either alphabet, as for the literal field of contents objects). For
the openpgp mechanism this is the binary signature (i.e. without
ASCII armor). For
pkcs7 this is the binary signature in DER.
The cryptographic signature MUST cover the exact, complete binary data of the JSON-SEQ sequence up to (but not including) the ASCII RS (0x1e) byte immediately preceding the first signature object.
If multiple signatures are provided, a tool processing manifest files may freely pick one of them to authenticate the manifest file, and ignore all others. Which one it picks is up to the tool, but SHOULD typically be dependent on locally supported mechanisms and recognized keys.
Note that the payload the signatures cover includes only the objects preceding the first signature object in the sequence, and excludes all signature objects and the trailer object.
Trailer Objects#
A manifest MUST contain a single trailer object. This object MUST appear in the JSON-SEQ sequence as the very last object, and terminates the sequence. It only has a single field:
{
"mediaType" : "application/vnd.uapi.16.trailer"
}Parsers MUST terminate once this object is seen, and fail if there is any further data (after the LF character that terminates the JSON-SEQ object) left in the byte stream. If a parser encounters a manifest without the trailer object it MUST assume the file is incomplete and generate an appropriate parsing failure.
Fields#
The mediaType is set to application/vnd.uapi.16.trailer for
objects of this type. This field MUST be used to recognize the
trailer object, and distinguish it from a file or signature object.
Compression#
Optionally, manifest files MAY be compressed with mechanisms such as
gzip, zstd or similar. In this case the manifest filename when
placed within the directory it covers SHOULD be Uapi16Manifest.gz,
Uapi16Manifest.zst and so on, as appropriate for the chosen
algorithm. If stored independently of the directory it covers the file
suffix should be .Uapi16Manifest.gz, .Uapi16Manifest.zst and so on.
Such compressed manifest files MUST be decompressed before validating the embedded signatures (as described above).
Relationship to SHA256SUMS#
A UAPI.16 manifest file that only contains the top-level dir file
object, dir file objects that only use the fields name and type,
and any number of reg file objects – each of which only uses the
fields name and sha256 – may be mapped to a SHA256SUMS file and
back. The SHA256SUMS file lists the reg file objects in the order of
the manifest. When mapping a SHA256SUMS file to a manifest, features
is set to ["sha256"] on the root file object, since a SHA256SUMS
file records nothing else, and a dir file object is added for every
directory its files are located in, right before the first file
object below it. This requires the files below every directory to be
listed together, which sorting them in the code point ordering of
complete names achieves. If no name contains a slash, the mapping is
1:1.
Acquiring a File Remotely#
When acquiring a file listed in a UAPI.16 File Manifest from a web service, the following logic should be implemented.
If
validityis set torevoked, the download SHOULD immediately fail.If
validSinceUSecis set and greater than the current time, the download SHOULD immediately fail.If
validUntilUSecis set and lower than the current time, the download SHOULD immediately fail.The
contentsarray MUST be iterated, and a suitable entry be picked as source for the file contents. If nocontentsarray is specified, proceed as in step 7, with an impliedfilefield set to thenameof the file object.If the selected contents object has
literalset, it MUST be Base64 decoded, continue in step 8.Otherwise, if
urlis set, the data from the URL MUST be acquired, including the additional HTTP header fields listed inhttpHeaders(if set) in the request. Continue in step 8.Otherwise, if
fileis set, the data MUST be acquired from the same location as the manifest itself, however with the last component of the location replaced by the file name encoded infile, following the same semantics as URI relative references. Continue in step 8.If
encodedSizeis set the size of the acquired data SHOULD be checked againstencodedSize. (This check – and similar checks below – may be omitted if for efficiency reasons the encoded file is only acquired partially.)If
encodedSha256is set, the hash of the acquired data SHOULD be matched against it.If
encodingis not set, butoriginalSizeis, the size of the acquired data SHOULD be checked againstoriginalSize.If
encodingis not set, butoriginalSha256is, the hash of the acquired data SHOULD be checked againstoriginalSha256.If
encodingis set, the acquired data MUST be decoded according to the algorithm indicated inencoding.If
encodingandoriginalSizeare set, the size of the resulting decoded data SHOULD be checked againstoriginalSize.If
encodingandoriginalSha256are set, the hash of the decoded data SHOULD be matched against it.The byte range indicated by
offset(if not set: 0) andsize(if not set, the remainder of the file) MUST be extracted from the decoded data.The hash of the extracted data MUST be matched against the hash encoded in
sha256(if set).
If all these steps succeed the extracted data from step 15 is the result of the operation.
Of course, many of the steps described above SHOULD typically be done
together rather than serially for robustness, efficiency and security
reasons. For example, if encoding is not used, it is recommended to
include the offset field (of the contents object) and the size
field (of the file object) in HTTP range
request fields already (in
order to avoid downloading redundant data). Moreover, downloads SHOULD
fail immediately once the encoded or decoded data as it is acquired
goes beyond the indicated encoded or original data sizes. Then, the
decoding SHOULD be done on-the-fly while the data is downloaded, and
the checksum should be calculated on-the-fly too.
Contents Source Considerations#
Applications may freely compile content object arrays when generating manifests, and are free to process the specified content object arrays in any order of preference when consuming them. A few suggestions:
An application might want to prefer placing file contents in
literalfields whenever the (encoded) file size is below some threshold, because embedding the data in the manifest (even if it is slightly increased in size because of that) might be more efficient in both space and processing time than referencing external resources that might need to be acquired in separate requests.An application might want to prefer not encoding (i.e. compressing) files below some size threshold, simply because the space savings the compression brings might not be worth the more expensive (in both space and processing time) acquisition of the resources.
When including file contents in
literalfields an application might choose not to add thesha256field of the same data, since it is redundant given that the literal data is also present in the manifest.When acquiring file contents from a manifest an application has freedom to prefer certain sources over others, or exclude some. For example, it is probably wise to prefer
literalsources over the others, as well as localfilesources over (remote)urlsources. It might also choose to retry acquiring the source with a different source of the specified ones if one does not work (for example because the source is not accessible or doesn’t actually have the resource in question). It might also prefer someurlsources over others, for example by determining that they are located closer on the network. It might also opt for not implementing theurlsource type at all, and instead requiringliteralorfilesources, depending on context.
Reproducibility#
This specification makes no attempt to define a single strictly reproducible profile of JSON or the manifest format. Its authors acknowledge that different JSON implementations will format objects and order fields differently depending on data, context and intended audience. Implementations may choose to include file metadata in varying detail in the manifests, for example UID/GID ownership might be relevant in some contexts and not in others. Because of that the cryptographic signatures are based on the actually formatted manifest file in the format the JSON library in use prefers, rather than a normalized representation of it.
Nonetheless implementations SHOULD generate manifests reproducibly for
the same input and the same selected fields – as long as the same
tooling in the same version is used. At the most basic this means
stable file object sorting should be employed, where no specific one
is mandated by this specification. The natural ordering of the
underlying file system SHOULD NOT be propagated into manifests
generated from it. Keys of the extendedAttributes object and of the
namespace objects SHOULD be sorted in the ordering of strings
(see above).
Extensibility#
Additional fields MAY be defined freely by implementors. Each such
extension field MUST be named in the style of x<Vendor>Foobar to
minimize risk of conflicts. For example xAmutableFrobnicate or
xMyAwesomeProjectEfficiencyVector. When reading, implementations MUST
ignore any fields they do not recognize.
Additional objects MAY be inserted at any place in the JSON-SEQ
sequence, except in front of the first (i.e. root file) object or
after the last (i.e. trailer) object. They MUST be marked via a
mediaType field, that describes what the object contains. When
processing a manifest file a reader MUST ignore (but SHOULD probably
log about) objects in the JSON-SEQ stream it does not recognize. Note
that any objects inserted before the first signature object will be
protected by the signature, but any objects inserted after will not
be. A tool processing manifest files MUST take this into account and
potentially ignore/refuse any objects appearing after the signatures
in the JSON sequence as unauthenticated data.
Future Revisions#
This specification is intended to be incrementally improved. Additional fields may be defined at any time, with additional, optional metadata. Should a breaking change be necessary the media type will be changed.
A number of future extensions are envisioned:
Fields for POSIX ACLs, and support for extended attribute namespaces beyond
userin theextendedAttributesfieldFields for additional extended file flags (further Linux
chattrflags, NTFS file system flags, …)Fields for Linux file security capabilities
Fields for security labels such as SMACK or SELinux
Additional hashing schemes
Additional signature schemes
Example#
<ASCII-RS>
{
"mediaType" : "application/vnd.uapi.16.manifest",
"features" : [ "gpt-label", "gpt-type-uuid", "gpt-uuid", "read-only", "sha256", "size" ]
}<ASCII-RS>
{
"name" : "FooOS.raw",
"contents" : [{
"file" : "FooOS.raw.gz",
"encoding" : "gzip",
"encodedSize" : 5642649603
}],
"size" : 7523532800,
"sha256" : "922a9bae0e02b4ffac3e5ed5054230d0689b9c2e25b0178ba82b925f2a0c3e48",
"validUntilUSec" : 1776856773123234
}<ASCII-RS>
{
"name" : "FooOS-esp.raw",
"contents" : [{
"file" : "FooOS.raw.gz",
"encoding" : "gzip",
"encodedSize" : 5642649603,
"originalSize" : 7523532800,
"offset" : 2097152
}],
"size" : 149175808,
"sha256" : "5dcfd837a4868550cc61c256d9567a974e32a20985afa9e100b8b96755a20cae",
"gptLabel" : "EFI System Partition",
"gptUuid" : "8a7f0bce-5fd7-4d23-9c12-3b6a4e1d9f07",
"gptTypeUuid" : "c12a7328-f81f-11d2-ba4b-00a0c93ec93b",
"validUntilUSec" : 1776856773123234
}<ASCII-RS>
{
"name" : "FooOS-root.raw",
"contents" : [{
"file" : "FooOS.raw.gz",
"encoding" : "gzip",
"encodedSize" : 5642649603,
"originalSize" : 7523532800,
"offset" : 351272960
}],
"size" : 234003200,
"sha256" : "00b70a0813e15f309828d3a36156283cba87576c26755e0b2d4cf0951eff8163",
"gptLabel" : "FooOS-root",
"gptUuid" : "1d2e3f40-51a6-4b78-9c0d-2e4f6a8b1c3d",
"gptTypeUuid" : "4f68bce3-e8cd-4db1-96e7-fbcaf984b709",
"readOnly" : true,
"validUntilUSec" : 1776856773123234
}<ASCII-RS>
{
"mediaType" : "application/vnd.uapi.16.signature",
"mechanism" : "openpgp",
"key" : "16b1c4eec0bc021ac777f681b63b21879c3485b0",
"data" : "iQIzBAABCgAdFiEEFrHE7sC8AhrHd/aBtjshh5w0hbAFAmn8rUoACgkQtjshh5w0hbAz3g//eyWO1swKXivio1PH2oJclEAYqpyqkv9LC293ePyDp5K0zL7BEVt9ijCGb22Wcu+xBXug4SBpSsCMOcro633Kn0ACBeOHwsyu1rnumb85/xvx3lSiq7ke1HcP5OWh5XX4lTm3KYVaPLQy0P3ThupJJsCa0o7uDdl2fXEgcZtRGhgp7GnpQabEuUUGW+1YZ1DFhhIdewZ6+eZZac0DPFjevBbI14jf9wEp3uWT9YoULF0wWBUi45vxdievwIRbcQHQVhxKMaTm0yuTbT5kSUAZccsOkJ4gqrVoNmFx0kYPve0uWcEkFQ16zEKVESr/fhDuCW19v3D46JNvwxZgIxP3o5rp3wbOja6n5D8Otl0kUQaE7FPOmYI1SZ2F97cYsKNViuUyrMcPTNGkI4E18H4WgWLf1CEyGHT16KmoCmldqVhQiTFCtOzkJdbwJgnSL/f0n2y3H/iB3lL0hglv2SDDVnVeMBx+qs+j2DMys3SVOOAYqGrMay1PYxLTEct+M1D0aAmH+XObuoVcm3XMtPPt6NCzxes0cdmFCdMTGERYLY522YCruJpO7p4h9IEoKqDACot0vDD0qwnjBCPu7ETrb02VvJ/W3GJ7/3Y8lIvZFtJqs5TQuxU0d4OiDHe08XdwZrcAondrX5h3Up7mpQIIF4RAMqUaYdOWEf7WzFZBKvY="
}<ASCII-RS>
{
"mediaType" : "application/vnd.uapi.16.trailer"
}The above defines three separate files FooOS.raw, FooOS-esp.raw
and FooOS-root.raw. All files may be acquired via the same data file
FooOS.raw.gz (which itself is however not listed in the manifest
as a file object). Note that the first defined file covers the whole
disk image, while the other two cover slices of it, each encapsulating
an individual partition. The data file is encoded via gzip.
A file object of a regular file carrying extended attributes in the
user namespace might look like this:
{
"name" : "notes.txt",
"mode" : 420,
"sha256" : "…",
"extendedAttributes" : {
"user" : {
"blob" : { "base64" : "AAECAw==" },
"mime_type" : { "text" : "text/plain" },
"xdg.origin.url" : { "text" : "https://example.com/notes.txt" }
}
}
}