GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — llama.cpp : add documentation about rope_freq_base and scale values by slaren · Pull Request #3401 · ggml-org/llama.cpp

Conversation

@slaren

@slaren slaren commented Sep 29, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Previously, setting rope_freq_base to 10000 and rope_freq_scale to 1 in llama_context_params would cause llama.cpp to use model's default value. Now, to use the model default values, these parameters must be set to zero.

Setting rope_freq_base to 10000 will cause problems with models trained on a different value, such as CodeLlama-7B. Therefore, downstream users should be careful to update the default values of these parameters.

When using llama_context_default_params() to initialize llama_context_params without changing these parameters, no further action is required. llama_context_default_params() will correctly set the value of these parameters to zero.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@slaren
llama.cpp : add documentation about rope_freq_base and scale values
2486725
@slaren

slaren commented Sep 29, 2023

Copy link
Copy Markdown
Member Author

I noticed that this is an issue in llama-cpp-python: abetlen/llama-cpp-python#765

Downstream users should be careful to update the default values.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ggerganov
ggerganov approved these changes Sep 29, 2023
View reviewed changes

@ggerganov ggerganov left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also add a hot topics entry given the impact and importance of this change

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
slaren added 3 commits September 29, 2023 18:18
@slaren
add notice to hot topics
1e3781c
@slaren
Update README.md
6d80a03
@slaren
Update README.md
777dae5
@slaren
slaren merged commit 40e07a6 into master Sep 29, 2023
@slaren
slaren deleted the cparams-doc branch September 29, 2023 16:42
This was referenced Sep 29, 2023
Change defaults for ROPE scaling Josh-XT/AGiXT#1015
Merged
The default values for rope_freq_base and rope_freq_scale override the model values abetlen/llama-cpp-python#765
Closed
Fix rope scaling defaults abetlen/llama-cpp-python#767
Merged
joelkuiper added a commit to vortext/llama.cpp that referenced this pull request Oct 2, 2023
@joelkuiper
Merge branch 'master' of github.com:ggerganov/llama.cpp into grammar-…
27d6181
…example

* 'master' of github.com:ggerganov/llama.cpp:
  ggml-cuda : perform cublas mat mul of quantized types as f16 (ggml-org#3412)
  llama.cpp : add documentation about rope_freq_base and scale values (ggml-org#3401)
  train : fix KQ_pos allocation (ggml-org#3392)
  llama : quantize up to 31% faster on Linux and Windows with mmap (ggml-org#3206)
  readme : update hot topics + model links (ggml-org#3399)
  readme : add link to grammars app (ggml-org#3388)
  swift : fix build on xcode 15 (ggml-org#3387)
  build : enable more non-default compiler warnings (ggml-org#3200)
  ggml_tensor: update the structure comments. (ggml-org#3283)
  ggml : release the requested thread pool resource (ggml-org#3292)
  llama.cpp : split llama_context_params into model and context params (ggml-org#3301)
  ci : multithreaded builds (ggml-org#3311)
  train : finetune LORA (ggml-org#2632)
  gguf : basic type checking in gguf_get_* (ggml-org#3346)
  gguf : make token scores and types optional (ggml-org#3347)
  ci : disable freeBSD builds due to lack of VMs (ggml-org#3381)
  llama : custom attention mask + parallel decoding + no context swaps (ggml-org#3228)
  docs : mark code as Bash (ggml-org#3375)
  readme : add Mistral AI release 0.1 (ggml-org#3362)
  ggml-cuda : perform cublas fp16 matrix multiplication as fp16 (ggml-org#3370)
jhen0409 added a commit to mybigday/llama.rn that referenced this pull request Oct 3, 2023
@jhen0409
fix(android): default rope_freq_base / rope_freq_scale to 0 during re…
43e036e
…cent breaking change

ref: ggml-org/llama.cpp#3401
yusiwen pushed a commit to yusiwen/llama.cpp that referenced this pull request Oct 7, 2023
@slaren @yusiwen
llama.cpp : add documentation about rope_freq_base and scale values (g…
0d04abb
…gml-org#3401)

* llama.cpp : add documentation about rope_freq_base and scale values

* add notice to hot topics
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
@slaren
llama.cpp : add documentation about rope_freq_base and scale values (g…
355794c
…gml-org#3401)

* llama.cpp : add documentation about rope_freq_base and scale values

* add notice to hot topics
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
@slaren
llama.cpp : add documentation about rope_freq_base and scale values (g…
a18aa62
…gml-org#3401)

* llama.cpp : add documentation about rope_freq_base and scale values

* add notice to hot topics
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
@slaren
llama.cpp : add documentation about rope_freq_base and scale values (g…
aecf248
…gml-org#3401)

* llama.cpp : add documentation about rope_freq_base and scale values

* add notice to hot topics
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
@slaren
llama.cpp : add documentation about rope_freq_base and scale values (g…
93e55e1
…gml-org#3401)

* llama.cpp : add documentation about rope_freq_base and scale values

* add notice to hot topics
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
@slaren
llama.cpp : add documentation about rope_freq_base and scale values (g…
99fdd10
…gml-org#3401)

* llama.cpp : add documentation about rope_freq_base and scale values

* add notice to hot topics
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@slaren @ggerganov
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.