API

The in-toto API provides various functions and classes that you can use to generate, consume, modify and verify in-toto metadata, as a more feature-rich, programmable alternative to the command line tools.

Note

Cryptographic Signatures

in-toto metadata is signed with cryptographic keys via the Signer API in securesystemslib Please refer to its documentation for details about loading a Signer for “Evidence Generation”.

Loading verification keys for “Supply Chain Verification” is documented in-toto#663.

Evidence Generation

in_toto.runlib.in_toto_run(name, material_list, product_list, link_cmd_args, record_streams=False, gpg_keyid=None, gpg_use_default=False, gpg_home=None, exclude_patterns=None, base_path=None, compact_json=False, record_environment=False, normalize_line_endings=False, lstrip_paths=None, metadata_directory=None, use_dsse=False, timeout=10, signer=None)

Performs a supply chain step or inspection generating link metadata.

Executes link_cmd_args, recording paths and hashes of files before and after command execution (aka. artifacts) in a link metadata file. The metadata is signed with the passed signer, a gpg key identified by its ID, or the default gpg key. If multiple key arguments are passed, only one key is used in above order of precedence. The resulting link file is written to STEP-NAME.KEYID-PREFIX.link. If no key argument is passed the link metadata is neither signed nor written to disk.

Parameters:

name – A unique name to associate link metadata with a step or inspection.
material_list – A list of artifact paths to be recorded before command execution. Directories are traversed recursively.
product_list – A list of artifact paths to be recorded after command execution. Directories are traversed recursively.
link_cmd_args – A list where the first element is a command and the remaining elements are arguments passed to that command.
record_streams (optional) – A boolean indicating if standard output and standard error of the link command should be recorded in the link metadata in addition to being displayed while the command is executed.
gpg_keyid (optional) – A keyid used to identify a local gpg key used to sign the resulting link metadata.
gpg_use_default (optional) – A boolean indicating if the default gpg key should be used to sign the resulting link metadata.
gpg_home (optional) – A path to the gpg home directory. If not set the default gpg home directory is used.
exclude_patterns (optional) – A list of filename patterns to exclude certain files from being recorded as artifacts. See Config docs for details.
base_path (optional) – A path relative to which artifacts are recorded. Default is the current working directory.
compact_json (optional) – A boolean indicating if the resulting link metadata should be written in the most compact JSON representation.
record_environment (optional) – A boolean indicating if information about the environment should be added in the resulting link metadata.
normalize_line_endings (optional) – A boolean indicating if line endings of artifacts should be normalized before hashing for cross-platform support.
lstrip_paths (optional) – A list of path prefixes used to left-strip artifact paths before storing them in the resulting link metadata.
metadata_directory (optional) – A directory path to write the resulting link metadata file to. Default destination is the current working directory.
use_dsse (optional) – A boolean indicating if DSSE should be used to generate metadata.
timeout (optional) – An integer indicating the max timeout in seconds for this command. Default is 10 seconds.
signer (optional) – A securesystemslib Signer instance used to sign the resulting link metadata.

Raises:

securesystemslib.exceptions.FormatError – Passed arguments are malformed.
OSError – Cannot change to base path directory.
securesystemslib.exceptions.StorageError – Cannot hash artifacts.
PrefixError – Left-stripping artifact paths results in non-unique dict keys.
subprocess.TimeoutExpired – Link command times out.
IOError, FileNotFoundError, NotADirectoryError, PermissionError – Cannot write link metadata.
securesystemslib.exceptions.CryptoError, securesystemslib.exceptions.UnsupportedAlgorithmError – Signing errors.
ValueError, OSError, securesystemslib._gpg.exceptions.CommandError, securesystemslib._gpg.exceptions.KeyNotFoundError – gpg signing errors.

Side Effects:

Reads artifact files from disk. Runs link command in subprocess. Calls system gpg in a subprocess, if a gpg key argument is passed. Writes link metadata file to disk, if any key argument is passed.

Returns:

A Metadata object that contains the resulting link object.

in_toto.runlib.in_toto_record_start(step_name, material_list, gpg_keyid=None, gpg_use_default=False, gpg_home=None, exclude_patterns=None, base_path=None, record_environment=False, normalize_line_endings=False, lstrip_paths=None, use_dsse=False, signer=None)

Generates preliminary link metadata.

Records paths and hashes of materials in a preliminary link metadata file. The metadata is signed with the passed signer, a gpg key identified by its ID, or the default gpg key. If multiple key arguments are passed, only one key is used in above order of precedence. At least one key argument must be passed. The resulting link file is written to .STEP-NAME.KEYID-PREFIX.link-unfinished.

Use this function together with in_toto_record_stop as an alternative to in_toto_run, in order to provide evidence for supply chain steps that cannot be carried out by a single command.

Parameters:

step_name – A unique name to associate link metadata with a step.
material_list – A list of artifact paths to be recorded as materials. Directories are traversed recursively.
gpg_keyid (optional) – A keyid used to identify a local gpg key used to sign the resulting link metadata.
gpg_use_default (optional) – A boolean indicating if the default gpg key should be used to sign the resulting link metadata.
gpg_home (optional) – A path to the gpg home directory. If not set the default gpg home directory is used.
exclude_patterns (optional) – A list of filename patterns to exclude certain files from being recorded as artifacts. See Config docs for details.
base_path (optional) – A path relative to which artifacts are recorded. Default is the current working directory.
record_environment (optional) – A boolean indicating if information about the environment should be added in the resulting link metadata.
normalize_line_endings (optional) – A boolean indicating if line endings of artifacts should be normalized before hashing for cross-platform support.
lstrip_paths (optional) – A list of path prefixes used to left-strip artifact paths before storing them in the resulting link metadata.
use_dsse (optional) – A boolean indicating if DSSE should be used to generate metadata.
signer (optional) – A securesystemslib Signer instance used to sign the resulting link metadata.

Raises:

securesystemslib.exceptions.FormatError – Passed arguments are malformed.
ValueError – None of signer, gpg_keyid or gpg_use_default=True is passed.
securesystemslib.exceptions.StorageError – Cannot hash artifacts.
PrefixError – Left-stripping artifact paths results in non-unique dict keys.
subprocess.TimeoutExpired – Link command times out.
IOError, PermissionError – Cannot write link metadata.
securesystemslib.exceptions.CryptoError, securesystemslib.exceptions.UnsupportedAlgorithmError – Signing errors.
ValueError, OSError, securesystemslib._gpg.exceptions.CommandError, securesystemslib._gpg.exceptions.KeyNotFoundError – gpg signing errors.

Side Effects:

Reads artifact files from disk. Calls system gpg in a subprocess, if a gpg key argument is passed. Writes preliminary link metadata file to disk.

in_toto.runlib.in_toto_record_stop(step_name, product_list, gpg_keyid=None, gpg_use_default=False, gpg_home=None, exclude_patterns=None, base_path=None, normalize_line_endings=False, lstrip_paths=None, metadata_directory=None, command=None, byproducts=None, environment=None, signer=None)

Finalizes preliminary link metadata generated with in_toto_record_start.

Loads preliminary link metadata file, verifies its signature, and records paths and hashes as products, thus finalizing the link metadata. The metadata is signed with the passed signer, a gpg key identified by its ID, or the default gpg key. If multiple key arguments are passed, only one key is used in above order of precedence. At least one key argument must be passed and it must be the same as the one used to sign the preliminary link metadata file. The resulting link file is written to STEP-NAME.KEYID-PREFIX.link.

Use this function together with in_toto_record_start as an alternative to in_toto_run, in order to provide evidence for supply chain steps that cannot be carried out by a single command.

Parameters:

step_name – A unique name to associate link metadata with a step.
product_list – A list of artifact paths to be recorded as products. Directories are traversed recursively.
gpg_keyid (optional) – A keyid used to identify a local gpg key used to sign the resulting link metadata.
gpg_use_default (optional) – A boolean indicating if the default gpg key should be used to sign the resulting link metadata.
gpg_home (optional) – A path to the gpg home directory. If not set the default gpg home directory is used.
exclude_patterns (optional) – A list of filename patterns to exclude certain files from being recorded as artifacts.
base_path (optional) – A path relative to which artifacts are recorded. Default is the current working directory.
normalize_line_endings (optional) – A boolean indicating if line endings of artifacts should be normalized before hashing for cross-platform support.
lstrip_paths (optional) – A list of path prefixes used to left-strip artifact paths before storing them in the resulting link metadata.
metadata_directory (optional) – A directory path to write the resulting link metadata file to. Default destination is the current working directory.
command (optional) – A list consisting of a command and arguments executed between in_toto_record_start() and in_toto_record_stop() to capture the command ran in the resulting link metadata.
byproducts (optional) – A dictionary that lists byproducts of the link command execution. It should have at least the following entries “stdout” (str), “stderr” (str) and “return-value” (int).

environment (optional) –

A dictionary to capture information about the environment to be added in the resulting link metadata eg.:

{
  "variables": "<list of env var KEY=value pairs>",
  "filesystem": "<filesystem info>",
  "workdir": "<CWD when executing link command>"
}

signer (optional) – A securesystemslib Signer instance used to sign the resulting link metadata.

Raises:

securesystemslib.exceptions.FormatError – Passed arguments are malformed.
ValueError – None of gpg_keyid or gpg_use_default=True is passed.
LinkNotFoundError – No preliminary link metadata file found.
securesystemslib.exceptions.StorageError – Cannot hash artifacts.
PrefixError – Left-stripping artifact paths results in non-unique dict keys.
subprocess.TimeoutExpired – Link command times out.
IOError, FileNotFoundError, NotADirectoryError, PermissionError – Cannot write link metadata.
securesystemslib.exceptions.CryptoError, securesystemslib.exceptions.UnsupportedAlgorithmError – Signing errors.
ValueError, OSError, securesystemslib._gpg.exceptions.CommandError, securesystemslib._gpg.exceptions.KeyNotFoundError – gpg signing errors.

Side Effects:

Reads preliminary link metadata file from disk. Reads artifact files from disk. Calls system gpg in a subprocess, if a gpg key argument is passed. Writes resulting link metadata file to disk. Removes preliminary link metadata file from disk.

in_toto.runlib.in_toto_match_products(link, paths=None, exclude_patterns=None, lstrip_paths=None)

Check if local artifacts match products in passed link.

NOTE: Does not check integrity or authenticity of passed link!

Parameters:: link – The Link object to match.

See in_toto_run for details about arguments, and exceptions that may occur while recording artifact hashes.

Returns:: A 3-tuple with artifact names that are - only in products, - not in products, - have different hashes.

Supply Chain Verification

in_toto.verifylib.in_toto_verify(metadata, layout_key_dict, link_dir_path='.', substitution_parameters=None, step_name='', persist_inspection_links=True, inspect_timeout=10)

Performs complete in-toto supply chain verification for a final product.

The verification procedure consists of the following activities, performed in the given order:

Verify layout signatures
Verify layout expiration date
Substitute placeholders in the layout
Load link metadata
Verify link signatures with keys in layout
Recurse into sublayout verification
Soft-verify alignment of reported and expected commands. Note: verification does not fail if commands don’t match, instead a warning is logged as per the in-toto specification – https://github.com/in-toto/docs/blob/master/in-toto-spec.md#431-steps
Verify threshold artifact constraints
Process step product and material rules
Execute inspection commands and generate inspection links
Process inspection product and material rules

Note: in-toto’s verification workflow directly uses the key for verifying signatures. It does not communicate with external sources of information for specific attributes about keys. This ensures that verification can be performed in isolation. It is the supply chain owner’s responsibility to consider such attributes before using the key to sign the layout or adding it to the layout. For example, in-toto does not rely on the creation time, revocation status, and usage flags for PGP keys. To revoke or otherwise affect the usage of a key, the supply chain owner must sign a new layout with the corresponding changes.

Parameters:

metadata – A Metadata object that contains a Layout object to be verified.
layout_key_dict – A public key dictionary. The verification routine requires at least one key, and a valid signature on the layout for each key.
link_dir_path (optional) – A directory path to link metadata files. The expected filename format for link metadata files is STEP-NAME.KEYID-PREFIX.link. Link metadata files for a sublayout are loaded from a subdirectory relative to the link_dir_path of the superlayout. The expected directory name format is SUBLAYOUT-STEP-NAME.KEYID-PREFIX.
substitution_parameters (optional) – A dictionary with substitution values for artifact rules (steps and inspections), the expected command attribute (steps), and the run attribute (inspections) in the layout.
step_name (optional) – A name assigned to the returned link. This is mostly useful during recursive sublayout verification.
persist_inspection_links (optional) – A boolean that determines whether or not link metadata files for inspection are written to cwd.
inspect_timeout (optional) – An integer value that defaults to in_toto.settings.LINK_CMD_EXEC_TIMEOUT in seconds which ends up timing out the run command subprocess if it runs over.

Raises:

securesystemslib.exceptions.FormatError – Passed parameters are malformed.
SignatureVerificationError – No layout verification key is passed, or any of the passed keys fails to verify a signature.
LayoutExpiredError – The layout is expired.
LinkNotFoundError – Fewer than threshold link metadata files can be found for a step of the layout.
ThresholdVerificationError – Fewer than threshold links, validly signed by different authorized functionaries, who agree on the recorded materials and products, can be found for a step of the layout. (Links with invalid signatures or signatures by unauthorized functionaries are ignored.)
RuleVerificationError – A DISALLOW rule matches disallowed artifacts, or a REQUIRE rule does not find a required artifact.
BadReturnValueError – An inspection command returns a non-zero value.

Side Effects:

Reads link metadata files from disk. Runs inspection commands in subprocess.

Returns:

A Link object, which summarizes the materials and products of the overall software supply chain. This is mostly useful during recursive sublayout verification.