Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update llama_factory.rst which might be more user-friendly for beginners #850

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

SuperAZHE
Copy link

I have a suggestion to enhance the script's description, which might be more user-friendly for beginners, reducing the level of confusion

I have a suggestion to enhance the script's description, which might be more user-friendly for beginners, reducing the level of confusion
@jklj077
Copy link
Collaborator

jklj077 commented Aug 19, 2024

IMO, enabling fa2 by default is prone to problems. however, I agree that there should be section on how fa2 works in llama-factory that includes at least:

  1. what kinds of devices are required
  2. what improvements should be expected (if users are with pytorch like 2.3 and flash_attn=auto in llama_factory, there are chances that fa2 is already enabled through sdpa).

@jklj077
Copy link
Collaborator

jklj077 commented Aug 19, 2024

ping @yangjianxin1 who is also working on this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants