19 lines
354 B
ReStructuredText
19 lines
354 B
ReStructuredText
.. _apiactivation:
|
|
|
|
flashinfer.activation
|
|
=====================
|
|
|
|
.. currentmodule:: flashinfer.activation
|
|
|
|
This module provides a set of activation operations for up/gate layers in transformer MLPs.
|
|
|
|
Up/Gate output activation
|
|
-------------------------
|
|
|
|
.. autosummary::
|
|
:toctree: ../generated
|
|
|
|
silu_and_mul
|
|
gelu_tanh_and_mul
|
|
gelu_and_mul
|