An implementation in Megatron-LM #12

Open

opened

on Apr 27, 2026

For anyone who wants to reproduce, I've implemented one based on Megatron-LM, with llama3 dense arch pretrained 1k steps, see https://github.com/Smu-Tan/Megatron-LM

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests