Fetch from nvidia Megatron-LM by RaymondLi0 · Pull Request #5 · ElementAI/Megatron-LM

RaymondLi0 · 2022-08-03T20:32:25Z

No description provided.

Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: mathemakitten <helen.ngo14@gmail.com> Co-authored-by: Helen Ngo <helenn@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@cw-dfw-cs-001-login-01.cm.cluster> Co-authored-by: shanmugamr1992 <shanmugamr1992@gmail.com>

Co-authored-by: Teodor-Dumitru Ene <teodord.ene@gmail.com> Co-authored-by: Robert Kirby <rkirby@nvidia.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@cw-dfw-cs-001-login-01.cm.cluster> Co-authored-by: rislam <rislam@nvidia.com>

…3799) Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

…" (#3834)

Co-authored-by: Keshav Santhanam <ksanthanam@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Salman Muin Kayser Chishti <13schishti@gmail.com> Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Salman Muin Kayser Chishti <13schishti@gmail.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: dimapihtar <dpihtar@gmail.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

…aph when running eager (#3815)

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@cw-dfw-cs-001-login-01.cm.cluster> Co-authored-by: shanmugamr1992 <shanmugamr1992@gmail.com>

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

…ed (#3851) Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com> Co-authored-by: gautham-kollu <gkollu@nvidia.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

…torch.distributed.checkpoint.state_dict_loader.load_state_dict (#2864)

…4133) Signed-off-by: Cory Ye <cye@nvidia.com>

Co-authored-by: William Dykas <wdykas@oci-hsg-cs-001-vscode-03.cm.cluster> Co-authored-by: root <root@nvl72077-T12.cm.cluster> Co-authored-by: root <root@nvl72098-T17.cm.cluster> Co-authored-by: root <root@nvl72160-T18.cm.cluster> Co-authored-by: root <root@nvl72006-T11.cm.cluster> Co-authored-by: root <root@nvl72074-T12.cm.cluster>

…yer for DiT (diffusion transformer) (#4134) Co-authored-by: Huy Vu2 <huvu@login-eos02.eos.clusters.nvidia.com>

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

…lel (#4259) Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

#4270)

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Co-authored-by: Li Tao <lit@nvidia.com>

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

…r__ (#3753) Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: gautham-kollu <gkollu@nvidia.com>

Signed-off-by: Hollow Man <hollowman@opensuse.org>

#4014) Co-authored-by: gautham-kollu <gkollu@nvidia.com>

…4281) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-authored-by: gautham-kollu <gkollu@nvidia.com>

Signed-off-by: Xin Yao <xiny@nvidia.com>

Signed-off-by: qiyuw <qiyuw@nvidia.com>

Signed-off-by: Deyu Fu <deyuf@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Signed-off-by: Xin Yao <xiny@nvidia.com>

ko3n1g force-pushed the main branch from ad5a627 to cd71d4c Compare February 18, 2026 09:18

jenchen13 and others added 29 commits March 11, 2026 18:32

Add NVIDIA-Nemotron-3-Super-120B-A12B-BF16 to ModelOpt examples (#3805)

5bc89f3

build: Bump TE2.13 (#3800)

da46946

Signed-off-by: oliver könig <okoenig@nvidia.com>

Ensure dummy_forward does not attempt to run cudagraphs (#3789)

8e64e69

Co-authored-by: mathemakitten <helen.ngo14@gmail.com> Co-authored-by: Helen Ngo <helenn@nvidia.com>

Add speculative decoding support with MTP layers (#3594)

8f539df

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@cw-dfw-cs-001-login-01.cm.cluster> Co-authored-by: shanmugamr1992 <shanmugamr1992@gmail.com>

Shanmugamr1992/megatron inference ultra (#3784)

39472d8

Co-authored-by: Teodor-Dumitru Ene <teodord.ene@gmail.com> Co-authored-by: Robert Kirby <rkirby@nvidia.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@cw-dfw-cs-001-login-01.cm.cluster> Co-authored-by: rislam <rislam@nvidia.com>

Fix backward compatibility issue with MFSDP --grad-reduce-in-bf16 (#…

d997820

…3799) Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

feat: add NCCL flight recorder configuration support (#3806)

251a754

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Revert "Ensure dummy_forward does not attempt to run cudagraphs (#3789)…

5a3aa17

…" (#3834)

Fix if statement in main (#3833)

5b25326

Co-authored-by: Keshav Santhanam <ksanthanam@nvidia.com>

Update golden values of weekly tests (#3829)

6657173

Signed-off-by: oliver könig <okoenig@nvidia.com>

build: Loosen TE restriction (#3827)

4736aed

Signed-off-by: oliver könig <okoenig@nvidia.com>

Upgrade GitHub Actions for Node 24 compatibility (#3830)

1d5e68b

Signed-off-by: Salman Muin Kayser Chishti <13schishti@gmail.com> Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Salman Muin Kayser Chishti <13schishti@gmail.com>

Do not let chunked prefill generate decode logprobs (#3777)

46227e0

Prevent double serialization inside Flask server (#3653)

e08dc9d

Allow RL to run inference-only via skip-train (#3744)

29e798a

Announce Python 3.12 migration (#3825)

4c1d0e4

Signed-off-by: oliver könig <okoenig@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

fabbcdf

ci: Skip test_wrong_cuda_graph_impl_returns_false in LTS (#3847)

b250472

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

ci: Mark TestCoordinator.test_throughput as flaky (#3849)

7ca9dc5

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

find optimal number of workers (#3699)

f261906

Signed-off-by: dimapihtar <dpihtar@gmail.com>

remove encoder_and_decoder (#3836)

b7437fe

Signed-off-by: dimapihtar <dpihtar@gmail.com>

ci: Skip more tests in test_vision_cuda_graphs for LTS (#3860)

8a806e5

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Ensure that inference dummy_forward does not try to match on a cudagr…

90fee1f

…aph when running eager (#3815)

Add unit tests for speculative decoding (#3817)

d1b8e27

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@cw-dfw-cs-001-login-01.cm.cluster> Co-authored-by: shanmugamr1992 <shanmugamr1992@gmail.com>

Fix flakiness due to timing between shutdowns (#3857)

d4ac04f

Exposing interleave argument for fused_apply_rotary_pos_emb_thd (#3794)

87eb3c2

ci: install nvidia-resiliency-ext from source (#3861)

9a19203

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Charlie Truong <chtruong@nvidia.com>

Miscellaneous inference bug fixes (#3840)

f8becec

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Nemo-RL integration bugfixes for --transformer-impl inference_optimiz…

905c0e3

…ed (#3851) Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

rkteddy and others added 30 commits April 10, 2026 13:24

Fix TransformerConfig validation for mixed dense/MoE upcycling (#3647)

e8e79a4

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com> Co-authored-by: gautham-kollu <gkollu@nvidia.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Remove cross-rank synchronization during checkpoint load & deprecate …

0602523

…torch.distributed.checkpoint.state_dict_loader.load_state_dict (#2864)

Fix incorrectly set decoupled_grad and DistOpt mechanics for MFSDP. (#…

ab43d43

…4133) Signed-off-by: Cory Ye <cye@nvidia.com>

Add conditions_embeddings argument to TransformerBlock, TransformerLa…

1daa19f

…yer for DiT (diffusion transformer) (#4134) Co-authored-by: Huy Vu2 <huvu@login-eos02.eos.clusters.nvidia.com>

Update copy-pr-bot.yaml [skip ci]

705d8ed

Fix build_sequences_per_dataset output path arg usage (#4144)

59fc894

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

ci: Flush pending CUDA work before the barrier in destroy_model_paral…

7cbc68c

…lel (#4259) Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Update oncall schedule (#4257)

e3d1204

docs(moe): Update MoE README (#3664)

41fcaa4

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Revert "Add conditions_embeddings argument to TransformerBlock, Trans… (

cc4cb01

#4270)

reduce the number of shared expert streams (#3752)

6da6267

remove legacy Bert code (#4204)

7f8f37e

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

[Main] Feat(moe): Gated delta net context parallel (CP) (#2642)

20ba03f

Co-authored-by: Li Tao <lit@nvidia.com>

remove t5 legacy code (#4203)

81b8b5a

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

fix: handle list-typed process groups in ProcessGroupCollection.__rep…

dda6901

…r__ (#3753) Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: gautham-kollu <gkollu@nvidia.com>

Fix Context Parallelism documentation link (#4149)

10f5fbd

[MLA] fix: Pad V when Q/V head dims differ for THD (#3003)

5dcda19

Signed-off-by: Hollow Man <hollowman@opensuse.org>

Allow the evaluation batch size to differ from the training batch size (

d85365b

#4014) Co-authored-by: gautham-kollu <gkollu@nvidia.com>

fix(megatron-fsdp): build expt_device_mesh only for MoE models (#3831)

25129bf

Add @NVIDIA/transformer review group to megatron/core/transformer/ (#…

a1595b8

…4281) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Reset AG_pipeline bucket status after validation step. (#3155)

5f80f0a

Co-authored-by: gautham-kollu <gkollu@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

e32e323

Enhance and fix NVTX for training (#3642)

df929f5

Signed-off-by: Xin Yao <xiny@nvidia.com>

NVFP4 native weights for DDP (#4005)

e1db4a0

Signed-off-by: qiyuw <qiyuw@nvidia.com>

Remove unnecessary arguments for layerwise distributed optimizer (#4272)

1ca250c

Signed-off-by: Deyu Fu <deyuf@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

reuse grad buffer for layer-wise param allgather (#3751)

28e13c4

feat(ci): add strict review mode to Claude review workflow (#4197)

e9e513a

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Fix stale approvals (#4280)

cd03c3e

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

[MoE] Add a new score function to the router (#3673)

eb80b74

Signed-off-by: Xin Yao <xiny@nvidia.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch from nvidia Megatron-LM#5

Fetch from nvidia Megatron-LM#5
RaymondLi0 wants to merge 7063 commits intoElementAI:load-iterfrom
NVIDIA:main

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants