Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

swift : improvements and fixes #3564

Merged
merged 3 commits into from
Oct 10, 2023
Merged

swift : improvements and fixes #3564

merged 3 commits into from
Oct 10, 2023

Conversation

jhen0409
Copy link
Collaborator

Some small improvements and fixes:

@zshannon
Copy link
Contributor

👍 was just coming here to ask about -O3 flag

@ggerganov ggerganov merged commit 0aa6595 into ggerganov:master Oct 10, 2023
38 checks passed
@l3utterfly
Copy link
Contributor

l3utterfly commented Oct 11, 2023 •
edited
Loading

I tried adding this before on my branch before, but after this change, I am unable to link it to a production iOS app. The error is: swift package cannot be used as a dependency because it contains unsafe build flags

So this change will stop this package from being used in production apps.

(Not sure if I'm doing something wrong)

@zshannon
Copy link
Contributor

There's something wonky with Xcode (surprise surprise) and SPM here where it throws that error if you're not pinned to a commit sha IIRC, could that be going on in your project config?

(The latest code doesn't work without -O3 so it's not like removing the flag is an option)

@l3utterfly
Copy link
Contributor

Huh, I'm pretty new to XCode so didn't know that. Can you give me an example on how to solve this by pinning it to a commit?

I'm using CMake for the time being.

@jhen0409 jhen0409 deleted the swift-improvements branch October 11, 2023 09:58
@jhen0409
Copy link
Collaborator Author

jhen0409 commented Oct 11, 2023 •
edited
Loading

@l3utterfly, In my remember this only happened if use local package, -fno-objc-arc and other flags will also cause this problem even if no -O3.

If you're using remote repository, set commit or version will be best practice, this can be done in Project -> Package Dependencies. However, even if I set master branch, I'm not able to get the error.

@l3utterfly
Copy link
Contributor

ah ok, I was using local packages. Thanks.

@jhen0409 were you able to test if this is still happening after enabling -O3? #3527

Your last EDIT said llama_save_session_file still has an issue.

@jhen0409
Copy link
Collaborator Author

jhen0409 commented Oct 12, 2023 •
edited
Loading

@jhen0409 were you able to test if this is still happening after enabling -O3? #3527

Your last EDIT said llama_save_session_file still has an issue.

I think llama_save_session_file is still broken with O3, currently I use it in main queue as a workaround.

@ggerganov
Copy link
Owner

Should be fixed on master via b8fe4b5

joelkuiper added a commit to vortext/llama.cpp that referenced this pull request Oct 12, 2023
@joelkuiper
Merge branch 'master' of github.com:ggerganov/llama.cpp into grammar-…
9871266 
…example

* 'master' of github.com:ggerganov/llama.cpp: (34 commits)
  examples: support LLaVA v1.5 (multimodal model) (ggerganov#3436)
  docs : fix typo GOMP_CPU_AFFINITY (ggerganov#3597)
  cmake : fix add_compile_options on macOS
  typo : it is `--n-gpu-layers` not `--gpu-layers` (ggerganov#3592)
  ci : check if there is enough VRAM (ggerganov#3596)
  server : add completion mode (no chat) (ggerganov#3582)
  prompts : add mnemonics.txt
  server : fix kv cache management (ggerganov#3588)
  main : fix session loading bug (ggerganov#3400)
  server : add parameter -tb N, --threads-batch N (ggerganov#3584)
  common : fix mirostat state when using multiple sequences (ggerganov#3543)
  batched : add bench tool (ggerganov#3545)
  examples : add batched.swift + improve CI for swift (ggerganov#3562)
  Add MPT model to supported models in README.md (ggerganov#3574)
  Minor improvements in GPT2 tokenizer (ggerganov#3567)
  readme : add bloom (ggerganov#3570)
  llm : add bloom models (ggerganov#3553)
  swift : improvements and fixes (ggerganov#3564)
  llm : add MPT support (ggerganov#3417)
  infill. : fix tokenization (ggerganov#3508)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jhen0409 @zshannon @l3utterfly @ggerganov