Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fused Rotary Embeddings (fixed) #1108

Merged
merged 29 commits into from
Jan 5, 2024
Merged

Fused Rotary Embeddings (fixed) #1108

merged 29 commits into from
Jan 5, 2024

Conversation

yang
Copy link
Contributor

@yang yang commented Jan 2, 2024 •
edited by StellaAthena
Loading

Orig PR started by Stella:

This is the start of an implementation of fused rotary embeddings for GPT-NeoX #1077. I assumed that it would be pretty straight forward despite my lack of familiarity with CUDA but I've been disabused of that notion.

Currently the issue is that there are errors when trying to compile the cuda extension.

StellaAthena and others added 27 commits November 15, 2023 11:38
@StellaAthena
Create fused_rotary_positional_embedding.cpp
4001ce1
@StellaAthena
Create fused_rotary_positional_embedding.h
54848a0
@StellaAthena
Create fused_rotary_positional_embedding_cuda.cu
e63242d
@StellaAthena
Update fused_rotary_positional_embedding.h
aaa000c 
Ports the fix from NVIDIA/apex#1750 into this branch.
@StellaAthena
Update neox_args.py
4808ced
@StellaAthena
Update setup.py
6e8e92a
@StellaAthena
Update initialize.py
eed2204
@StellaAthena
Update setup.py
6ff6180
@StellaAthena
Update __init__.py
0bd0839
@StellaAthena
Update test_fused_kernels.py
11a3798
@StellaAthena
Update setup.py
8bf4bc4
@StellaAthena
Create fused_rope.py
d86c399
@StellaAthena
Update fused_rotary_positional_embedding.h
417f55c
@StellaAthena
Update fused_rotary_positional_embedding.cpp
5b1331d
@StellaAthena
Update fused_rotary_positional_embedding.cpp
f20f1f5
@StellaAthena
Merge pull request EleutherAI#1085 from EleutherAI/avoid-.contiguous()
af387b0 
Avoid .contiguous()
@StellaAthena
Update transformer.py
e35955b
@StellaAthena
Update transformer.py
fc214d0 
Just checked and this should work for bf16. Or, at least, the reason I originally thought it wouldn't doesn't apply.
@StellaAthena
Update transformer.py
f745997
@StellaAthena
Create 125M_fused_rope.yml
29f50ae
@StellaAthena
Update 125M_fused_rope.yml
491806b
@StellaAthena
Merge pull request EleutherAI#1089 from EleutherAI/main
feb1433 
update from main
@StellaAthena
Update transformer.py
589323d 
Add `self.rope_fusion = neox_args.rope_fusion` so that `ParallelSelfAttention` knows if we're using rope fusion.
Update NeoXArgs docs automatically
7f3ae33
@Quentin-Anthony
Merge branch 'main' into fused-rope
a3572c7
Update NeoXArgs docs automatically
59f6f3b
@yang
Fix fused rope
3058110 
Just needed to bring in the latest headers/sources,
and call into it the right way from transformers.py.
@yang yang requested a review from Quentin-Anthony as a code owner January 2, 2024 19:09
@StellaAthena StellaAthena linked an issue Jan 2, 2024 that may be closed by this pull request
Apply new fused rotary embedding #1077
Closed
@StellaAthena
Copy link
Member

StellaAthena commented Jan 2, 2024 •
edited
Loading

Supersedes #1103

@Quentin-Anthony Quentin-Anthony merged commit 77605ca into EleutherAI:main Jan 5, 2024
3 of 5 checks passed
@Quentin-Anthony Quentin-Anthony mentioned this pull request Jan 5, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Quentin-Anthony Quentin-Anthony Quentin-Anthony approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Apply new fused rotary embedding
3 participants
@yang @StellaAthena @Quentin-Anthony