Operators

Every operator below is declared with @operator on its module class; these pages are generated from those declarations by python -m hndl.docs.

activation

Operator	Summary
elu	Exponential linear unit: identity above zero, saturating below.
gelu	Gaussian error linear unit, x * Phi(x).
hardswish	Piecewise-linear approximation of swish, x * relu6(x + 3) / 6.
identity	Pass the tensor through unchanged, as a named node.
leaky_relu	ReLU with a small slope for negative inputs.
mish	Self-gated smooth activation, x * tanh(softplus(x)).
quick_gelu	CLIP's fast GELU approximation, x * sigmoid(1.702 * x).
relu	Rectified linear unit, max(x, 0).
sigmoid	Logistic sigmoid, squashing values into (0, 1).
silu	Sigmoid linear unit (swish), x * sigmoid(x).
softmax	Normalize one non-batch axis into a probability distribution.
softplus	Smooth positive activation, log(1 + exp(beta*x)) / beta.
tanh	Hyperbolic tangent, squashing values into (-1, 1).

arithmetic

Operator	Summary
add	Elementwise sum of two tensors with identical shapes.
broadcast_add	Elementwise sum of two tensors of equal rank, broadcasting size-1 axes.
clamp	Saturate every element into the closed interval [min, max].
constant	Emit a tensor of a fixed shape filled with one constant value.
learned_scale	Multiply a tensor by one learned scalar.
matmul	Batched matrix product of two rank-3 tensors.
mul	Elementwise product of two tensors with identical shapes.
scale	Multiply a tensor by a fixed scalar.
sub	Elementwise difference of two tensors with identical shapes.

convolution

Operator	Summary
conv	Two-dimensional convolution over [B, C, H, W] images.
conv1d	One-dimensional convolution over [B, C, L] signals.
deconv	Transposed two-dimensional convolution, typically for upsampling.

core

Operator	Summary
linear	Fully connected layer: a learned affine map on the last axis.

memory

Operator	Summary
hopfield	Retrieve learned patterns by iterating the modern Hopfield update on the last axis.

normalization

Operator	Summary
adaptive_norm	Instance-normalize features, then apply a per-example learned scale and bias.
batch_norm	Normalize each channel over the batch and spatial axes, tracking running statistics.
group_norm	Normalize channel groups per example, with learned per-channel affine.
instance_norm	Normalize every channel of every example over its own spatial positions.
layer_norm	Normalize the last axis of every position with a learned scale and bias.
rms_norm	Scale the last axis by its root-mean-square, with a learned per-feature gain.

pretrained

Operator	Summary
pretrained	Load a pretrained network from disk or the Hugging Face Hub as one frozen node.

regularization

Operator	Summary
dropout	Randomly zero elements during training and rescale the rest.

sequence

Operator	Summary
attention	Multi-head self-attention over a [B, T, D] sequence.
cls_token	Prepend one learned classification token to a sequence.
cross_attention	Multi-head attention with queries from one sequence and keys and values from another.
embedding	Look up a learned vector for every integer token id.
feed_forward	Transformer feed-forward block: widen, activate, project back.
gather_token	Select the token of a sequence at the position holding the largest id.
moe	Sparse mixture of experts: route every token to its top-k feed-forward experts.
pool_tokens	Reduce a [B, T, D] sequence to one [B, D] vector per example.
pos_embed	Add a learned position vector to every position of a sequence.
swiglu	SwiGLU feed-forward block: a SiLU-gated projection folded back to the input width.
transformer_block	Pre-norm transformer block: residual self-attention followed by a residual feed-forward.

shape

Operator	Summary
chunk	Cut one axis into a fixed number of equal sections.
concat	Join two or more tensors along one axis.
flatten	Collapse every non-batch axis into one feature axis.
mean	Average one non-batch axis away.
pad	Enlarge trailing axes with constant, reflected, or replicated borders.
permute	Reorder the non-batch axes, keeping the batch axis first.
reshape	View the tensor with new non-batch dimensions, preserving the element count.
split	Cut one axis into a first section and the remainder.
sum	Add up one non-batch axis.
transpose	Swap two non-batch axes of the tensor.

spatial

Operator	Summary
adaptive_avg_pool	Average-pool [B, C, H, W] images to a fixed output height and width.
avg_pool	Two-dimensional average pooling over [B, C, H, W] images.
global_avg_pool	Average each channel over height and width, producing one value per channel.
max_pool	Two-dimensional max pooling over [B, C, H, W] images.
pixel_shuffle	Trade channels for resolution: rearrange C*r^2 channels into an r-times larger image.
spatial_attention	SAGAN self-attention over every position of a [B, C, H, W] feature map.
upsample	Enlarge height and width by an integer factor with a fixed interpolation kernel.

vision

Operator	Summary
patch_embed	Cut an image into non-overlapping patches and embed each one as a token.
resblock	Residual basic block: two 3x3 convolutions with a normalized shortcut.