Subtests

Run independent blocks of a test so that one block failing still lets the others report their own result.

A production test often checks several unrelated functions of a product — Wi-Fi, Bluetooth, audio, display. Without subtests, the first failing block either stops the test (STOP) or is just another failed phase in a flat list. Subtests give each block a name and an outcome of its own, and FAIL_SUBTEST fails only that block.

Syntax

htf.Subtest(name, *phases) wraps a sequence. Names must be unique within a test (DuplicateSubtestNamesError otherwise). A failed measurement alone does not end a subtest — just as it does not end a plain test — so put a checkpoint with action=FAIL_SUBTEST after the phases that gate the rest of the block, or return FAIL_SUBTEST explicitly.

import openhtf as htf
from openhtf.core.phase_branches import PhaseFailureCheckpoint

@htf.measures(htf.Measurement("rssi_dbm").in_range(-70, 0))
def wifi_scan(test):
    test.measurements.rssi_dbm = -85          # Fails validation

def wifi_throughput(test):
    # Not reached: the checkpoint fails the subtest after wifi_scan
    return htf.PhaseResult.CONTINUE

@htf.measures(htf.Measurement("bt_paired").equals(True))
def bluetooth_pair(test):
    test.measurements.bt_paired = True

def main():
    test = htf.Test(
        htf.Subtest(
            "wireless_wifi",
            wifi_scan,
            PhaseFailureCheckpoint.subtest_previous(
                "wifi_ok", action=htf.PhaseResult.FAIL_SUBTEST),   # fail this subtest only
            wifi_throughput,
        ),
        htf.Subtest("wireless_bluetooth", bluetooth_pair),   # Still runs
    )
    test.execute(lambda: "SN1234")

if __name__ == "__main__":
    main()

======================= test: openhtf_test  outcome: FAIL ======================

The record's subtests[] shows wireless_wifi as FAIL and wireless_bluetooth as PASS; wifi_throughput is recorded with outcome SKIP.

FAIL_SUBTEST

PhaseResult.FAIL_SUBTEST marks the phase as failed and skips the remaining phases of the enclosing subtest (they are recorded with outcome SKIP); execution continues with the next node after the subtest. Outside a subtest it is treated as an ERROR.

def wifi_connect(test):
    if not try_connect():
        test.logger.error("No association after 3 attempts")
        return htf.PhaseResult.FAIL_SUBTEST
    return htf.PhaseResult.CONTINUE

Outcomes in the record

Each subtest is summarized in test_record.subtests[] with name, outcome (PASS, FAIL, STOP) and timestamps, and every phase inside carries subtest_name. The overall test outcome is FAIL if any subtest failed. See the JSON format reference.

Checkpoints inside subtests

Every checkpoint has an action, default PhaseResult.STOP — a plain checkpoints.checkpoint() inside a subtest stops the whole test, and the subtest is recorded with outcome STOP. Pass action=htf.PhaseResult.FAIL_SUBTEST to fail only the enclosing subtest and continue.

• PhaseFailureCheckpoint.subtest_previous(name, action=...) considers only the phases of the current subtest. Use it when earlier subtests are allowed to fail.
• PhaseFailureCheckpoint.all_previous(name, action=...) (what checkpoints.checkpoint() builds) considers every earlier phase in the test — including previous subtests — even with FAIL_SUBTEST.
• PhaseFailureCheckpoint.last(name, action=...) considers only the immediately preceding phase.

Subtests vs phase groups

They compose: a subtest for "Wi-Fi" can contain a phase group that always powers the radio down.

Related