ControlNet

ControlNet, created by Lvmin Zhang, is a neural network architecture that adds spatial conditioning to pretrained text-to-image diffusion models such as Stable Diffusion. It works by duplicating a network block into a locked copy that preserves the original weights and a trainable copy that learns the new condition, joined through zero convolution layers initialized to zero so training starts without distorting the base model's output. Supported conditioning inputs in the official release include Canny edges, HED boundaries, M-LSD lines, user scribbles, human pose skeletons, depth maps, normal maps, semantic segmentation, and anime line art, and multiple ControlNets can be composed for multi-condition control. A guess mode even generates images from the control signal alone without a text prompt. The openly released code and models reshaped image generation workflows, with implementations across Diffusers, ComfyUI, and the AUTOMATIC1111 WebUI, and ControlNet remains a core technique for artists and developers who need precise structural control over generated images.