colossalai.nn.layer

• colossalai.nn.layer.colossalai_layer
  • Linear
  • Classifier
  • Embedding
  • PatchEmbedding
  • LayerNorm
  • Dropout
  • colossalai.nn.layer.colossalai_layer.dropout
  • colossalai.nn.layer.colossalai_layer.embedding
  • colossalai.nn.layer.colossalai_layer.linear
  • colossalai.nn.layer.colossalai_layer.normalization
• colossalai.nn.layer.moe
  • FFNExperts
  • TPExperts
  • Top1Router
  • Top2Router
  • NormalNoiseGenerator
  • UniformNoiseGenerator
  • MoeModule
  • MoeRouter
  • colossalai.nn.layer.moe.experts
  • colossalai.nn.layer.moe.layers
  • colossalai.nn.layer.moe.utils
• colossalai.nn.layer.parallel_1d
  • Linear1D
  • Linear1D_Col
  • Linear1D_Row
  • Embedding1D
  • Dropout1D
  • Classifier1D
  • VocabParallelClassifier1D
  • VocabParallelEmbedding1D
  • LayerNorm1D
  • PatchEmbedding1D
  • colossalai.nn.layer.parallel_1d.layers
• colossalai.nn.layer.parallel_2d
  • split_batch_2d()
  • reduce_by_batch_2d()
  • Linear2D
  • LayerNorm2D
  • Classifier2D
  • PatchEmbedding2D
  • Embedding2D
  • VocabParallelEmbedding2D
  • VocabParallelClassifier2D
  • colossalai.nn.layer.parallel_2d.layers
• colossalai.nn.layer.parallel_2p5d
  • split_batch_2p5d()
  • reduce_by_batch_2p5d()
  • Linear2p5D
  • LayerNorm2p5D
  • Classifier2p5D
  • PatchEmbedding2p5D
  • Embedding2p5D
  • VocabParallelClassifier2p5D
  • VocabParallelEmbedding2p5D
  • colossalai.nn.layer.parallel_2p5d.layers
• colossalai.nn.layer.parallel_3d
  • reduce_by_batch_3d()
  • split_tensor_3d()
  • split_batch_3d()
  • Linear3D
  • LayerNorm3D
  • PatchEmbedding3D
  • Classifier3D
  • Embedding3D
  • VocabParallelEmbedding3D
  • VocabParallelClassifier3D
  • colossalai.nn.layer.parallel_3d.layers
• colossalai.nn.layer.parallel_sequence
  • TransformerSelfAttentionRing
  • RingAV
  • RingQK
  • colossalai.nn.layer.parallel_sequence.layers
• colossalai.nn.layer.utils
  • divide()
  • colossalai.nn.layer.utils.common
• colossalai.nn.layer.vanilla
  • VanillaLayerNorm
  • VanillaPatchEmbedding
  • VanillaClassifier
  • DropPath
  • WrappedDropout
  • WrappedDropPath
  • VanillaLinear
  • colossalai.nn.layer.vanilla.layers
• colossalai.nn.layer.wrapper
  • colossalai.nn.layer.wrapper.pipeline_wrapper

• colossalai.nn.layer.base_layer