HLO (High Level Operations)
HLO, expanded as High Level Operations and also as High Level Optimizer, is the intermediate representation and operation set at the centre of XLA, the machine-learning compiler developed at Google and now maintained within the OpenXLA ecosystem. It models a computation as a graph over multidimensional arrays using a small, closed set of typed primitive operations, on which the compiler performs target-independent and target-dependent optimisation before lowering to native code for CPUs, GPUs and accelerator hardware such as the Tensor Processing Unit.
The term denotes both the compiler's internal IR — a hand-written representation with its own text and protobuf formats, deliberately not built on MLIR — and the family of high-level tensor operations that is standardised across frameworks as StableHLO, the versioned MLIR dialect that acts as a portability layer between machine-learning frameworks and compilers. Around these sit related dialects such as CHLO, MHLO, LMHLO and VHLO, and a toolchain that includes XLA, the PJRT device API and the MLIR infrastructure. Because "HLO" is also an ambiguous initialism used in maritime, administrative and transport contexts, this wiki concerns the machine-learning compiler sense and treats the other meanings only insofar as they bound that scope.
HLO emerged with XLA's announcement in March 2017, when Google sought to reconcile TensorFlow's flexibility with the performance of custom accelerators. It grew alongside MLIR and the MLIR-HLO project before being superseded by StableHLO and consolidated under OpenXLA in 2023. Its practical importance lies in enabling a single compiler to serve many frameworks and many hardware backends, to fuse operation sequences aggressively, to plan memory statically, and to partition large models across thousands of devices. Its limitations — compilation cost, inflexibility with dynamic shapes and the opacity of custom operations — are the subject of continuing debate.