-
Notifications
You must be signed in to change notification settings - Fork 9.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Model agnostic DAN prompt #1304
Conversation
This seems extremely verbose for what it does. Between the long kv calculation, the lack of GPT-3 style refusals to cooperate in most models training sets, and the limited context space without flash attention I'd be a little surprised if you are actually seeing better performance with this.
Do you have examples of models refusing prompts that prepending this fixes?
|
Just giving it few-shot "illegal" answers isn't enough? |
Just to mention, the original DAN prompt is already in the repository (https://github.com/ggerganov/llama.cpp/blob/master/prompts/dan.txt), so it's not like this is without precedent. But it's also worth noting that someone could whine if we directly distributed illegal content within a prompt, where DAN just pulls it out of the user supplied model. |
Since the model is running locally, there's a much simpler approach that works really well. Just fill in part of the LLM's response with something that commits it to answering. I.E. prompt in Vicuna 1.1 format:
and it started generating:
If you actually want to find out how to hotwire a car you can ask it yourself, I won't include the whole response here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
drop the changes from the README.md
This reverts commit 8dc342c.
Related to comments in #1270