colossalai.utils.memory_tracer.model_data_memtracer

class colossalai.utils.memory_tracer.model_data_memtracer.ModelDataTracer(*args, **kwargs)

A tracer singleton to trace model data usage during runtime. The tracer is designed to trace the memory layout change during model-data tensors allocation, releasing, and moving. To achieve this goal, the developers have to call ModelDataTracer in the corresponding code explicitly. NOTE() now the class only trace cuda memory usage