colossalai.context

This module is serving for receiving and passing user’s configuration to all devices to initialize and construct parallel training.

• colossalai.context.process_group_initializer
  • colossalai.context.process_group_initializer.initializer_1d
  • colossalai.context.process_group_initializer.initializer_2d
  • colossalai.context.process_group_initializer.initializer_2p5d
  • colossalai.context.process_group_initializer.initializer_3d
  • colossalai.context.process_group_initializer.initializer_data
  • colossalai.context.process_group_initializer.initializer_model
  • colossalai.context.process_group_initializer.initializer_pipeline
  • colossalai.context.process_group_initializer.initializer_sequence
  • colossalai.context.process_group_initializer.initializer_tensor
  • colossalai.context.process_group_initializer.process_group_initializer
• colossalai.context.random
  • colossalai.context.random.seed_manager

• colossalai.context.config
• colossalai.context.parallel_context
• colossalai.context.moe_context
• colossalai.context.parallel_mode