Output Callbacks

Learn how to create structured test results with OpenHTF, covering JSON output and custom output mechanisms with detailed examples.

Last updated · Verified with OpenHTF 1.6.1

Enable automatic test report generation in JSON or other formats.

When the test ends, OpenHTF passes the finished test record to every registered output callback. The built-in JSON callback writes a file named after the device under test; a custom callback can send the record anywhere.

  • Test end
  • TestRecord
  • OutputToJSON("./{dut_id}.json")
  • custom callbackdef upload(record): ...

Tracking tests is crucial for traceability and analysis, especially in production environments. OpenHTF uses output callbacks to automatically generate reports, with a default JSON format that can be customized as needed.

JSON

You can enable automatic test result export in the default JSON format.

main.py
from openhtf import Test, PhaseResult
from openhtf.output.callbacks import json_factory

def get_sn(test):
    test.test_record.dut_id = 'SN1234'
    return PhaseResult.CONTINUE

def main():
    test = Test(get_sn)
    # Exports results to JSON with pretty-printing
    test.add_output_callbacks(json_factory.OutputToJSON("test_result.json", indent=2))
    test.execute()

if __name__ == "__main__":
    main()

A test_result.json file is created after execution and saved in the execution directory. The pattern can use any field of the record, so one file per run is the usual choice:

json_factory.OutputToJSON("./records/{dut_id}.{metadata[test_name]}.{start_time_millis}.json", indent=2)
filename_pattern_or_filestr, callable or file

Format string expanded with the record's fields, a callable returning a path, or an open file object.

indentint or None

Pretty-print indentation. None writes compact JSON.

inline_attachmentsbool

Default True: attachments are base64-encoded into the JSON. Set False to keep the file small when phases attach images or long logs.

The general structure of an OpenHTF JSON record is as follows (full field list in the JSON format reference):

  dut_id
  start_time_millis
  end_time_millis
  outcome                    # PASS/FAIL
  metadata
    └── test_name
    └── config
    └── ...
  phases                     # Array of test phases executed
    └── name
    └── outcome              # PASS/FAIL
    └── result               # Phase-specific result details (e.g., CONTINUE)
    └── measurements
  log_records                # Array of log messages
    └── message
    └── timestamp_millis
  station_id
  code_info                  # Details about the test script (name, source code)

Console summary

ConsoleSummary prints the outcome and, on failure, every failed measurement with its value and validator. Useful while developing a test:

main.py
import openhtf as htf
from openhtf.output.callbacks import console_summary

@htf.measures(htf.Measurement("voltage").in_range(3.0, 3.6))
def measure(test):
    test.measurements.voltage = 4.1

def main():
    test = htf.Test(measure)
    test.add_output_callbacks(console_summary.ConsoleSummary())
    test.execute(lambda: "SN1234")

if __name__ == "__main__":
    main()
Terminal
openhtf_test:FAIL
failed phase: measure [ran for 0.00 sec]
  failed_item: voltage (Outcome.FAIL)
    measured_value: 4.1
    validators:
      validator: 3.0 <= x <= 3.6

======================= test: openhtf_test  outcome: FAIL ======================

Other built-ins: callbacks.OutputToFile(pattern) pickles the record, and mfg_inspector.MfgInspector converts it to Google's TestRun protobuf (save_to_disk() / upload()). Both live under openhtf.output.callbacks.

Send records to a database

Output callbacks run for every test, which makes them the right place to push records to a database instead of (or as well as) a file. TofuPilot's client is an output callback:

main.py
from tofupilot.openhtf import upload

test.add_output_callbacks(upload())   # reads TOFUPILOT_API_KEY from the environment

Records land in a workspace with yield and Cpk per measurement — see Manufacturing Test Analytics. The same pattern fits any REST API or database driver, as the custom callback below shows.

Custom output format

You can implement a custom output format by creating a function to handle the test record and adding it as an output callback if the built-in format doesn't meet your needs. Callbacks run in order after the test finishes.

main.py
import openhtf as htf

def custom_output_callback(test_record):
    with open("./custom_output.txt", "w") as f:
        f.write("Custom Output\n")
        f.write(f"DUT ID: {test_record.dut_id}\n")
        f.write(f"Outcome: {test_record.outcome}\n")
        for phase in test_record.phases:
            f.write(f"Phase: {phase.name}\n")

def get_sn(test):
    test.test_record.dut_id = "SN1234"
    return htf.PhaseResult.CONTINUE

def main():
    test = htf.Test(get_sn)
    test.add_output_callbacks(custom_output_callback)
    test.execute()

if __name__ == "__main__":
    main()

The output file will then be:

custom_output.txt
Custom Output
DUT ID: SN1234
Outcome: Outcome.PASS
Phase: get_sn

Test Record

The object every callback receives.

JSON format reference

Field-by-field description of the file.

Ignore early-canceled tests

Subclass OutputToJSON to filter records.

Diagnoses

Use OpenHTF diagnoses to analyze results after measurement validation, classify failure modes, and drive conditional execution between phases.

Test Record

Complete reference for the OpenHTF test record structure including test outcomes, phase records, measurement data, attachments, and diagnosis results.

On this page

JSONConsole summarySend records to a databaseCustom output formatRelated
First-pass yield
0%4.1
Track with TofuPilot