Skip to content

[FEATURE] Support for RWKV and MAMBA architecture #2363

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2catycm opened this issue Dec 12, 2024 · 1 comment
Open

[FEATURE] Support for RWKV and MAMBA architecture #2363

2catycm opened this issue Dec 12, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@2catycm
Copy link

2catycm commented Dec 12, 2024

Is your feature request related to a problem? Please describe.
RWKV and MAMBA are new families of models that are efficient. It would be great if TIMM lib have this

Describe the solution you'd like
add a new file in timm to implement them.

Describe alternatives you've considered

Additional context
Here is the official lib of Vision-RWKV https://github.com/OpenGVLab/Vision-RWKV , and here is the paper https://openreview.net/forum?id=nGiGXLnKhl . And for RWKV this is the website https://rwkv.cn/ .

As for mamba, there are a lot of vision implementations. For example

@2catycm 2catycm added the enhancement New feature or request label Dec 12, 2024
@rwightman
Copy link
Collaborator

@2catycm they are both interesting model families, the problem is they all require custom kernels (or external libraries with custom kernels). Those have proven to be difficult to maintain over time, other models requiring custom kernels have come and gone in the time frame timm has existed and original authors did not maintain those kernels for current iterations of cuda/torch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants