OpenHTF vs Robot Framework
Compare OpenHTF and Robot Framework for hardware and manufacturing tests — keyword-driven .robot suites vs Python phases, who writes the tests, measurements and limits, instrument libraries, operator interaction and reporting.
Last updated · Verified with OpenHTF 1.6.1
Robot Framework and OpenHTF both come up when a hardware team looks beyond home-grown scripts. Robot is a generic, keyword-driven automation framework with a large ecosystem; OpenHTF is a Python framework built specifically for testing physical units.
One-line answer
Choose Robot Framework when non-programmers must write and read the test steps, or when the same keyword libraries already drive your system tests. Choose OpenHTF when the people writing the test are engineers comfortable in Python and the output must be measurements with limits per unit.
Side by side
| OpenHTF | Robot Framework | |
|---|---|---|
| Test definition | Python functions with decorators | .robot files: tabular keywords, Gherkin optional |
| Who writes tests | Test/hardware engineers in Python | Anyone, once keywords exist; keyword authors in Python |
| Measurements | First-class: value, limits, unit, marginal, stored per run | Not built in; Should Be True ${x} > 4.8 loses the value unless logged |
| Hardware access | Plugs with lifecycle | Keyword libraries (Python classes) with suite/test setup |
| Ordering & flow | Sequential, PhaseResult, groups, checkpoints, branches, subtests | Sequential within a suite; Run Keyword If, tags |
| Operator UI | Built in | None built in |
| DUT / serial tracking | dut_id on every record | Variables you define |
| Reports | JSON test record per run + optional Operator UI history | output.xml, log.html, report.html per execution |
| Ecosystem | Small, hardware-focused | Large: web, API, mobile, RPA libraries |
| Data model consumer | Manufacturing databases, SPC | Test-management and CI dashboards |
Where the difference shows
Readability vs precision
*** Test Cases ***
Supply Voltage Is Stable
${voltage}= Read Voltage
Should Be True 4.8 <= ${voltage} <= 5.2 Supply voltage out of rangeReadable by a technician; the number is gone from the report unless a keyword logs it.
@htf.measures(htf.Measurement("supply_voltage").in_range(4.8, 5.2).with_units(units.VOLT))
def supply_voltage(test, dmm):
test.measurements.supply_voltage = dmm.read_voltage()Less approachable for a non-programmer; every run keeps the value, limits and unit.
Keyword libraries vs plugs
Both are Python classes wrapping instruments. A Robot library is instantiated per suite or per test and called through keyword names; an OpenHTF plug is instantiated per run, injected into phases, torn down after, and its logs land in the record. Porting between them is mechanical.
Reporting
Robot's log.html is excellent for one execution. It is not a per-unit database: comparing unit 4,000 with unit 12 means parsing many output.xml files. OpenHTF's record is one JSON per unit, built for that.
Where Robot Framework is the better tool
- Acceptance tests written by QA or product owners in plain language.
- System tests that mix hardware with web UIs, REST APIs and mobile apps using existing Robot libraries.
- Organisations already standardised on Robot's reports and test-management integrations.
Using both
TofuPilot runs Robot Framework suites on stations through the Listener API and ships a tofupilot_robot library whose Measure Numeric keyword keeps limits end to end — so a Robot suite can produce OpenHTF-style per-unit records. See Robot Framework on TofuPilot. For a new production test written by engineers, OpenHTF remains the shorter path.
Related
OpenHTF vs pytest
A practical comparison of OpenHTF and pytest for testing physical products — execution model, measurements vs assertions, fixtures vs plugs, operator interaction, output records — and when to use each or both.
OpenHTF vs TestStand
Compare OpenHTF with NI TestStand (and LabVIEW-based test systems) for manufacturing test — sequence editor vs Python code, licensing, instrument drivers, operator interfaces, reports and databases, and what a migration involves.
