Support of Grok1 Model #373

quic-amitraj · 2025-04-22T05:29:37Z

Support of Grok1 Model

QEfficient/transformers/models/grok_1/__init__.py

QEfficient/transformers/models/grok_1/modeling_grok1.py

quic-rishinr · 2025-06-05T10:07:36Z

Add the mode to the validated list and please post the accuracy numbers as well

QEfficient/base/pytorch_transforms.py

quic-akuruvil · 2025-06-06T06:28:46Z

Add the mode to the validated list and please post the accuracy numbers as well

Also, grok1 2-layer model needs to be added to test files, and please paste with this PR the perplexity numbers in torch and AIC.

QEfficient/transformers/models/grok_1/modeling_grok1.py

quic-amitraj · 2025-06-06T08:46:26Z

Add the mode to the validated list and please post the accuracy numbers as well

Added.

quic-amitraj · 2025-06-06T08:48:33Z

Add the mode to the validated list and please post the accuracy numbers as well

Also, grok1 2-layer model needs to be added to test files, and please paste with this PR the perplexity numbers in torch and AIC.

All

Output matches end to end on Pytorch, ONNX and AI 100 for single layer.

tests/transformers/models/test_causal_lm_models.py

ochougul

Is this tested via infer CLI command?

quic-akuruvil · 2025-06-07T03:06:08Z

Add the mode to the validated list and please post the accuracy numbers as well

Also, grok1 2-layer model needs to be added to test files, and please paste with this PR the perplexity numbers in torch and AIC.

All

Output matches end to end on Pytorch, ONNX and AI 100 for single layer.

This is good. But Anuj has suggested to do a first order perplexity analysis from our side for text models, before we close a model, and pass on to accuracy team. Can check with Anuj and run the perplexity scripts if needed.

quic-amitraj · 2025-06-09T05:05:49Z

Is this tested via infer CLI command?

It was not supported through infer as this model is not integrated with Transformers. With latest commit added support of this.

quic-hemagnih

Looks good to me, we can plan to merge the changes.

QEfficient/base/common.py

QEfficient/base/modeling_qeff.py

Signed-off-by: Amit Raj <[email protected]>

Signed-off-by: Onkar Chougule <[email protected]>

Signed-off-by: Amit Raj <[email protected]>

…te code execution Signed-off-by: Amit Raj <[email protected]>

Signed-off-by: Amit Raj <[email protected]>

Signed-off-by: Rishin Raj <[email protected]>

Signed-off-by: Abukhoyer Shaik <[email protected]>

quic-hemagnih · 2025-06-12T08:16:00Z

Closing this Pull request as we have migrated these changes to another PR: #447

quic-amitraj mentioned this pull request Apr 22, 2025

Support of Grok1ModelForCausalLM #360

Closed

quic-amitraj self-assigned this Apr 22, 2025

quic-amitraj added the model-enablement label Apr 22, 2025

quic-amitraj mentioned this pull request Apr 22, 2025

Feat: Onbaord PlamoForCausalLM Architecture #351

Draft

quic-amitraj marked this pull request as ready for review April 29, 2025 10:30

quic-amitraj requested review from quic-rishinr and ochougul as code owners April 29, 2025 10:30

quic-hemagnih reviewed Jun 4, 2025

View reviewed changes

QEfficient/transformers/models/grok_1/__init__.py Outdated Show resolved Hide resolved

quic-hemagnih reviewed Jun 4, 2025

View reviewed changes

QEfficient/transformers/models/grok_1/modeling_grok1.py Show resolved Hide resolved

quic-hemagnih reviewed Jun 4, 2025

View reviewed changes

QEfficient/transformers/models/grok_1/modeling_grok1.py Show resolved Hide resolved

quic-hemagnih reviewed Jun 5, 2025

View reviewed changes

QEfficient/transformers/models/grok_1/modeling_grok1.py Show resolved Hide resolved

quic-hemagnih reviewed Jun 5, 2025

View reviewed changes

QEfficient/transformers/models/grok_1/modeling_grok1.py Outdated Show resolved Hide resolved

quic-hemagnih reviewed Jun 5, 2025

View reviewed changes

QEfficient/transformers/models/grok_1/modeling_grok1.py Outdated Show resolved Hide resolved

quic-hemagnih reviewed Jun 5, 2025

View reviewed changes

QEfficient/transformers/models/grok_1/modeling_grok1.py Show resolved Hide resolved

quic-hemagnih added the 1.20.0 label Jun 5, 2025

quic-rishinr requested changes Jun 6, 2025

View reviewed changes

QEfficient/base/pytorch_transforms.py Show resolved Hide resolved

QEfficient/base/pytorch_transforms.py Show resolved Hide resolved

quic-hemagnih requested a review from quic-akuruvil June 6, 2025 05:52

quic-akuruvil reviewed Jun 6, 2025

View reviewed changes

QEfficient/transformers/models/grok_1/modeling_grok1.py Outdated Show resolved Hide resolved

quic-amitraj force-pushed the grok_new branch from a71f631 to a74b9a7 Compare June 6, 2025 09:46

ochougul reviewed Jun 6, 2025

View reviewed changes

tests/transformers/models/test_causal_lm_models.py Outdated Show resolved Hide resolved

ochougul requested changes Jun 6, 2025

View reviewed changes

quic-amitraj force-pushed the grok_new branch from fa9f098 to a33fb48 Compare June 9, 2025 05:23

quic-hemagnih approved these changes Jun 9, 2025

View reviewed changes

quic-rishinr reviewed Jun 9, 2025

View reviewed changes

QEfficient/base/common.py Outdated Show resolved Hide resolved

quic-amitraj force-pushed the grok_new branch from ffb7cab to 1889bd5 Compare June 9, 2025 11:09

quic-rishinr requested changes Jun 10, 2025

View reviewed changes

QEfficient/base/modeling_qeff.py Outdated Show resolved Hide resolved

quic-amitraj force-pushed the grok_new branch from 37e5a2d to b5abfaf Compare June 10, 2025 04:31

quic-rishinr approved these changes Jun 10, 2025

View reviewed changes

quic-amitraj force-pushed the grok_new branch from 39ccb4b to e4599ab Compare June 11, 2025 15:49

quic-amitraj and others added 14 commits June 11, 2025 15:53

Added support of Grok

b19945d

Signed-off-by: Amit Raj <[email protected]>

Used common code from llama to remove redundancy

6c170e4

Signed-off-by: Amit Raj <[email protected]>

Added RMS norm changes and MoE laltest changes

582faa2

Signed-off-by: Amit Raj <[email protected]>

Updated latest MOE changes

d369c85

Signed-off-by: Amit Raj <[email protected]>

added efficient MOE avoiding redudant reads in prefill for down weights

7d6fae7

Signed-off-by: Onkar Chougule <[email protected]>

Inital commit

2fa5d1d

Signed-off-by: Amit Raj <[email protected]>

Added test

613b1ad

Signed-off-by: Amit Raj <[email protected]>

Addressed comments and added support of the external model using Infer

066359c

Signed-off-by: Amit Raj <[email protected]>

Added new argument trust_remote_code in infer which will help for emo…

01fa36e

…te code execution Signed-off-by: Amit Raj <[email protected]>

Addressed comments

4a34ed9

Signed-off-by: Amit Raj <[email protected]>

Added the check if device is occupied in get_available_device_id API

eccd4c0

Signed-off-by: Rishin Raj <[email protected]>

Updated device check logic

3205dc3

Signed-off-by: Rishin Raj <[email protected]>

trust_remote_code enabled for grok1 only

ea0850d

Signed-off-by: Abukhoyer Shaik <[email protected]>

trust_remote_code enabled for grok1 only

4304fe9

Signed-off-by: Abukhoyer Shaik <[email protected]>

quic-amitraj force-pushed the grok_new branch from e4599ab to 4304fe9 Compare June 11, 2025 15:54

quic-hemagnih closed this Jun 12, 2025

Support of Grok1 Model #373

Support of Grok1 Model #373

Uh oh!

Conversation

quic-amitraj commented Apr 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

quic-rishinr commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

quic-akuruvil commented Jun 6, 2025

Uh oh!

Uh oh!

quic-amitraj commented Jun 6, 2025

Uh oh!

quic-amitraj commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ochougul left a comment

Choose a reason for hiding this comment

Uh oh!

quic-akuruvil commented Jun 7, 2025

Uh oh!

quic-amitraj commented Jun 9, 2025

Uh oh!

quic-hemagnih left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

quic-hemagnih commented Jun 12, 2025

Uh oh!

Uh oh!

quic-rishinr commented Jun 5, 2025 •

edited

Loading

quic-amitraj commented Jun 6, 2025 •

edited

Loading