GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||
GeezerVison is showing github.com — examples : add persistent chat by ejones · Pull Request #1495 · ggml-org/llama.cpp Conversation
Collaborator
Depends on #1032 (to test this, rebase on that first) Adds a chat-13B-style example that persists the chat session and leverages
Testing: Tested with 30B and the same chat as chat-13B. Exited, resumed (by rerunning same command), and crossed context swaps. Also started and resumed other chat sessions, using their own Approach
The ongoing chat transcript is persisted to
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page. All reactions
ggerganov
approved these changes
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🦙 !
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page. All reactions
Member
All reactions
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page.
Collaborator
Author
All reactions
Sorry, something went wrong. Uh oh!
There was an error while loading. Please reload this page.
ggerganov
added a commit
to JohannesGaessler/llama.cpp
that referenced
this pull request
* examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]>
ggerganov
added a commit
that referenced
this pull request
…oadcasting for ggml_mul (#1483) * Broadcasting for ggml_mul * CUDA kernel for ggml_mul, norms in VRAM * GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * define default model path once, sync path with readme (#1366) * ~7% faster Q5_1 AVX2 code (#1477) * convert.py: Support models which are stored in a single pytorch_model.bin (#1469) * Support models in a single pytorch_model.bin * Remove spurious line with typo * benchmark-matmul: Print the average of the test results (#1490) * Remove unused n_parts parameter (#1509) * Fixes #1511 lambda issue for w64devkit (mingw) (#1513) * Fix for w64devkit and mingw * make kv_f16 the default for api users (#1517) * minor : fix compile warnings * readme : adds WizardLM to the list of supported models (#1485) * main : make reverse prompt option act as a stop token in non-interactive mode (#1032) * Make reverse prompt option act as a stop token in non-interactive scenarios * Making requested review changes * Update gpt_params_parse and fix a merge error * Revert "Update gpt_params_parse and fix a merge error" This reverts commit 2bb2ff1. * Update gpt_params_parse and fix a merge error take 2 * examples : add persistent chat (#1495) * examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]> * tests : add missing header * ggml : use F16 instead of F32 in Q4_0, Q4_1, Q8_0 (#1508) * ggml : use F16 instead of F32 in Q4_0, Q4_1 and Q8_0 * llama : bump LLAMA_FILE_VERSION to 3 * cuda : update Q4 and Q8 dequantize kernels * ggml : fix AVX dot products * readme : update performance table + hot topics * ggml : fix scalar implementation of Q4_1 dot * llama : fix compile warnings in llama_set_state_data() * llama : fix name shadowing and C4146 (#1526) * Fix name shadowing and C4146 * Fix if macros not using defined when required * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Code style Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Georgi Gerganov <[email protected]> * Fix for mingw (#1462) * llama : add llama_init_backend() API (close #1527) * feature : add blis and other BLAS implementation support (#1502) * feature: add blis support * feature: allow all BLA_VENDOR to be assigned in cmake arguments. align with whisper.cpp pr 927 * fix: version detection for BLA_SIZEOF_INTEGER, recover min version of cmake * Fix typo in INTEGER Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: Georgi Gerganov <[email protected]> * Revert "feature : add blis and other BLAS implementation support (#1502)" This reverts commit 07e9ace. * GPU weights not in RAM, direct loading with cuFile * llama : code style fixes + progress print fix * ggml : ggml_mul better broadcast support * cmake : workarounds for cufile when CMake version < 3.25 * gg rebase fixup * Loop in llama.cpp, fixed progress callback * Attempt clang-tidy fix * llama : fix vram size computation * Add forgotten fclose() --------- Co-authored-by: András Salamon <[email protected]> Co-authored-by: Ilya Kurdyukov <[email protected]> Co-authored-by: Tom Jobbins <[email protected]> Co-authored-by: rankaiyx <[email protected]> Co-authored-by: Stephan Walter <[email protected]> Co-authored-by: DannyDaemonic <[email protected]> Co-authored-by: Erik Scholz <[email protected]> Co-authored-by: Georgi Gerganov <[email protected]> Co-authored-by: David Kennedy <[email protected]> Co-authored-by: Jason McCartney <[email protected]> Co-authored-by: Evan Jones <[email protected]> Co-authored-by: Maxime <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Zenix <[email protected]>
Seunghhon
pushed a commit
to Seunghhon/llama.cpp
that referenced
this pull request
* examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]>
Seunghhon
pushed a commit
to Seunghhon/llama.cpp
that referenced
this pull request
…oadcasting for ggml_mul (ggml-org#1483) * Broadcasting for ggml_mul * CUDA kernel for ggml_mul, norms in VRAM * GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * define default model path once, sync path with readme (ggml-org#1366) * ~7% faster Q5_1 AVX2 code (ggml-org#1477) * convert.py: Support models which are stored in a single pytorch_model.bin (ggml-org#1469) * Support models in a single pytorch_model.bin * Remove spurious line with typo * benchmark-matmul: Print the average of the test results (ggml-org#1490) * Remove unused n_parts parameter (ggml-org#1509) * Fixes ggml-org#1511 lambda issue for w64devkit (mingw) (ggml-org#1513) * Fix for w64devkit and mingw * make kv_f16 the default for api users (ggml-org#1517) * minor : fix compile warnings * readme : adds WizardLM to the list of supported models (ggml-org#1485) * main : make reverse prompt option act as a stop token in non-interactive mode (ggml-org#1032) * Make reverse prompt option act as a stop token in non-interactive scenarios * Making requested review changes * Update gpt_params_parse and fix a merge error * Revert "Update gpt_params_parse and fix a merge error" This reverts commit 2bb2ff1. * Update gpt_params_parse and fix a merge error take 2 * examples : add persistent chat (ggml-org#1495) * examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]> * tests : add missing header * ggml : use F16 instead of F32 in Q4_0, Q4_1, Q8_0 (ggml-org#1508) * ggml : use F16 instead of F32 in Q4_0, Q4_1 and Q8_0 * llama : bump LLAMA_FILE_VERSION to 3 * cuda : update Q4 and Q8 dequantize kernels * ggml : fix AVX dot products * readme : update performance table + hot topics * ggml : fix scalar implementation of Q4_1 dot * llama : fix compile warnings in llama_set_state_data() * llama : fix name shadowing and C4146 (ggml-org#1526) * Fix name shadowing and C4146 * Fix if macros not using defined when required * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Code style Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Georgi Gerganov <[email protected]> * Fix for mingw (ggml-org#1462) * llama : add llama_init_backend() API (close ggml-org#1527) * feature : add blis and other BLAS implementation support (ggml-org#1502) * feature: add blis support * feature: allow all BLA_VENDOR to be assigned in cmake arguments. align with whisper.cpp pr 927 * fix: version detection for BLA_SIZEOF_INTEGER, recover min version of cmake * Fix typo in INTEGER Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: Georgi Gerganov <[email protected]> * Revert "feature : add blis and other BLAS implementation support (ggml-org#1502)" This reverts commit 07e9ace. * GPU weights not in RAM, direct loading with cuFile * llama : code style fixes + progress print fix * ggml : ggml_mul better broadcast support * cmake : workarounds for cufile when CMake version < 3.25 * gg rebase fixup * Loop in llama.cpp, fixed progress callback * Attempt clang-tidy fix * llama : fix vram size computation * Add forgotten fclose() --------- Co-authored-by: András Salamon <[email protected]> Co-authored-by: Ilya Kurdyukov <[email protected]> Co-authored-by: Tom Jobbins <[email protected]> Co-authored-by: rankaiyx <[email protected]> Co-authored-by: Stephan Walter <[email protected]> Co-authored-by: DannyDaemonic <[email protected]> Co-authored-by: Erik Scholz <[email protected]> Co-authored-by: Georgi Gerganov <[email protected]> Co-authored-by: David Kennedy <[email protected]> Co-authored-by: Jason McCartney <[email protected]> Co-authored-by: Evan Jones <[email protected]> Co-authored-by: Maxime <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Zenix <[email protected]>
phuongncn
pushed a commit
to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4
that referenced
this pull request
* examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]>
phuongncn
pushed a commit
to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4
that referenced
this pull request
…oadcasting for ggml_mul (ggml-org#1483) * Broadcasting for ggml_mul * CUDA kernel for ggml_mul, norms in VRAM * GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * define default model path once, sync path with readme (ggml-org#1366) * ~7% faster Q5_1 AVX2 code (ggml-org#1477) * convert.py: Support models which are stored in a single pytorch_model.bin (ggml-org#1469) * Support models in a single pytorch_model.bin * Remove spurious line with typo * benchmark-matmul: Print the average of the test results (ggml-org#1490) * Remove unused n_parts parameter (ggml-org#1509) * Fixes ggml-org#1511 lambda issue for w64devkit (mingw) (ggml-org#1513) * Fix for w64devkit and mingw * make kv_f16 the default for api users (ggml-org#1517) * minor : fix compile warnings * readme : adds WizardLM to the list of supported models (ggml-org#1485) * main : make reverse prompt option act as a stop token in non-interactive mode (ggml-org#1032) * Make reverse prompt option act as a stop token in non-interactive scenarios * Making requested review changes * Update gpt_params_parse and fix a merge error * Revert "Update gpt_params_parse and fix a merge error" This reverts commit 2bb2ff1. * Update gpt_params_parse and fix a merge error take 2 * examples : add persistent chat (ggml-org#1495) * examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]> * tests : add missing header * ggml : use F16 instead of F32 in Q4_0, Q4_1, Q8_0 (ggml-org#1508) * ggml : use F16 instead of F32 in Q4_0, Q4_1 and Q8_0 * llama : bump LLAMA_FILE_VERSION to 3 * cuda : update Q4 and Q8 dequantize kernels * ggml : fix AVX dot products * readme : update performance table + hot topics * ggml : fix scalar implementation of Q4_1 dot * llama : fix compile warnings in llama_set_state_data() * llama : fix name shadowing and C4146 (ggml-org#1526) * Fix name shadowing and C4146 * Fix if macros not using defined when required * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Code style Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Georgi Gerganov <[email protected]> * Fix for mingw (ggml-org#1462) * llama : add llama_init_backend() API (close ggml-org#1527) * feature : add blis and other BLAS implementation support (ggml-org#1502) * feature: add blis support * feature: allow all BLA_VENDOR to be assigned in cmake arguments. align with whisper.cpp pr 927 * fix: version detection for BLA_SIZEOF_INTEGER, recover min version of cmake * Fix typo in INTEGER Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: Georgi Gerganov <[email protected]> * Revert "feature : add blis and other BLAS implementation support (ggml-org#1502)" This reverts commit 07e9ace. * GPU weights not in RAM, direct loading with cuFile * llama : code style fixes + progress print fix * ggml : ggml_mul better broadcast support * cmake : workarounds for cufile when CMake version < 3.25 * gg rebase fixup * Loop in llama.cpp, fixed progress callback * Attempt clang-tidy fix * llama : fix vram size computation * Add forgotten fclose() --------- Co-authored-by: András Salamon <[email protected]> Co-authored-by: Ilya Kurdyukov <[email protected]> Co-authored-by: Tom Jobbins <[email protected]> Co-authored-by: rankaiyx <[email protected]> Co-authored-by: Stephan Walter <[email protected]> Co-authored-by: DannyDaemonic <[email protected]> Co-authored-by: Erik Scholz <[email protected]> Co-authored-by: Georgi Gerganov <[email protected]> Co-authored-by: David Kennedy <[email protected]> Co-authored-by: Jason McCartney <[email protected]> Co-authored-by: Evan Jones <[email protected]> Co-authored-by: Maxime <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Zenix <[email protected]>
ljubomirj
pushed a commit
to ljubomirj/llama.cpp
that referenced
this pull request
* examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]>
ljubomirj
pushed a commit
to ljubomirj/llama.cpp
that referenced
this pull request
…oadcasting for ggml_mul (ggml-org#1483) * Broadcasting for ggml_mul * CUDA kernel for ggml_mul, norms in VRAM * GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * define default model path once, sync path with readme (ggml-org#1366) * ~7% faster Q5_1 AVX2 code (ggml-org#1477) * convert.py: Support models which are stored in a single pytorch_model.bin (ggml-org#1469) * Support models in a single pytorch_model.bin * Remove spurious line with typo * benchmark-matmul: Print the average of the test results (ggml-org#1490) * Remove unused n_parts parameter (ggml-org#1509) * Fixes ggml-org#1511 lambda issue for w64devkit (mingw) (ggml-org#1513) * Fix for w64devkit and mingw * make kv_f16 the default for api users (ggml-org#1517) * minor : fix compile warnings * readme : adds WizardLM to the list of supported models (ggml-org#1485) * main : make reverse prompt option act as a stop token in non-interactive mode (ggml-org#1032) * Make reverse prompt option act as a stop token in non-interactive scenarios * Making requested review changes * Update gpt_params_parse and fix a merge error * Revert "Update gpt_params_parse and fix a merge error" This reverts commit 2bb2ff1. * Update gpt_params_parse and fix a merge error take 2 * examples : add persistent chat (ggml-org#1495) * examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]> * tests : add missing header * ggml : use F16 instead of F32 in Q4_0, Q4_1, Q8_0 (ggml-org#1508) * ggml : use F16 instead of F32 in Q4_0, Q4_1 and Q8_0 * llama : bump LLAMA_FILE_VERSION to 3 * cuda : update Q4 and Q8 dequantize kernels * ggml : fix AVX dot products * readme : update performance table + hot topics * ggml : fix scalar implementation of Q4_1 dot * llama : fix compile warnings in llama_set_state_data() * llama : fix name shadowing and C4146 (ggml-org#1526) * Fix name shadowing and C4146 * Fix if macros not using defined when required * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Code style Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Georgi Gerganov <[email protected]> * Fix for mingw (ggml-org#1462) * llama : add llama_init_backend() API (close ggml-org#1527) * feature : add blis and other BLAS implementation support (ggml-org#1502) * feature: add blis support * feature: allow all BLA_VENDOR to be assigned in cmake arguments. align with whisper.cpp pr 927 * fix: version detection for BLA_SIZEOF_INTEGER, recover min version of cmake * Fix typo in INTEGER Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: Georgi Gerganov <[email protected]> * Revert "feature : add blis and other BLAS implementation support (ggml-org#1502)" This reverts commit 3516762. * GPU weights not in RAM, direct loading with cuFile * llama : code style fixes + progress print fix * ggml : ggml_mul better broadcast support * cmake : workarounds for cufile when CMake version < 3.25 * gg rebase fixup * Loop in llama.cpp, fixed progress callback * Attempt clang-tidy fix * llama : fix vram size computation * Add forgotten fclose() --------- Co-authored-by: András Salamon <[email protected]> Co-authored-by: Ilya Kurdyukov <[email protected]> Co-authored-by: Tom Jobbins <[email protected]> Co-authored-by: rankaiyx <[email protected]> Co-authored-by: Stephan Walter <[email protected]> Co-authored-by: DannyDaemonic <[email protected]> Co-authored-by: Erik Scholz <[email protected]> Co-authored-by: Georgi Gerganov <[email protected]> Co-authored-by: David Kennedy <[email protected]> Co-authored-by: Jason McCartney <[email protected]> Co-authored-by: Evan Jones <[email protected]> Co-authored-by: Maxime <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Zenix <[email protected]>
AlexiAlp
pushed a commit
to minghaop/llama.cpp
that referenced
this pull request
* examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]>
AlexiAlp
pushed a commit
to minghaop/llama.cpp
that referenced
this pull request
…oadcasting for ggml_mul (ggml-org#1483) * Broadcasting for ggml_mul * CUDA kernel for ggml_mul, norms in VRAM * GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * define default model path once, sync path with readme (ggml-org#1366) * ~7% faster Q5_1 AVX2 code (ggml-org#1477) * convert.py: Support models which are stored in a single pytorch_model.bin (ggml-org#1469) * Support models in a single pytorch_model.bin * Remove spurious line with typo * benchmark-matmul: Print the average of the test results (ggml-org#1490) * Remove unused n_parts parameter (ggml-org#1509) * Fixes ggml-org#1511 lambda issue for w64devkit (mingw) (ggml-org#1513) * Fix for w64devkit and mingw * make kv_f16 the default for api users (ggml-org#1517) * minor : fix compile warnings * readme : adds WizardLM to the list of supported models (ggml-org#1485) * main : make reverse prompt option act as a stop token in non-interactive mode (ggml-org#1032) * Make reverse prompt option act as a stop token in non-interactive scenarios * Making requested review changes * Update gpt_params_parse and fix a merge error * Revert "Update gpt_params_parse and fix a merge error" This reverts commit 2bb2ff1. * Update gpt_params_parse and fix a merge error take 2 * examples : add persistent chat (ggml-org#1495) * examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]> * tests : add missing header * ggml : use F16 instead of F32 in Q4_0, Q4_1, Q8_0 (ggml-org#1508) * ggml : use F16 instead of F32 in Q4_0, Q4_1 and Q8_0 * llama : bump LLAMA_FILE_VERSION to 3 * cuda : update Q4 and Q8 dequantize kernels * ggml : fix AVX dot products * readme : update performance table + hot topics * ggml : fix scalar implementation of Q4_1 dot * llama : fix compile warnings in llama_set_state_data() * llama : fix name shadowing and C4146 (ggml-org#1526) * Fix name shadowing and C4146 * Fix if macros not using defined when required * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Code style Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Georgi Gerganov <[email protected]> * Fix for mingw (ggml-org#1462) * llama : add llama_init_backend() API (close ggml-org#1527) * feature : add blis and other BLAS implementation support (ggml-org#1502) * feature: add blis support * feature: allow all BLA_VENDOR to be assigned in cmake arguments. align with whisper.cpp pr 927 * fix: version detection for BLA_SIZEOF_INTEGER, recover min version of cmake * Fix typo in INTEGER Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: Georgi Gerganov <[email protected]> * Revert "feature : add blis and other BLAS implementation support (ggml-org#1502)" This reverts commit 16cdcea. * GPU weights not in RAM, direct loading with cuFile * llama : code style fixes + progress print fix * ggml : ggml_mul better broadcast support * cmake : workarounds for cufile when CMake version < 3.25 * gg rebase fixup * Loop in llama.cpp, fixed progress callback * Attempt clang-tidy fix * llama : fix vram size computation * Add forgotten fclose() --------- Co-authored-by: András Salamon <[email protected]> Co-authored-by: Ilya Kurdyukov <[email protected]> Co-authored-by: Tom Jobbins <[email protected]> Co-authored-by: rankaiyx <[email protected]> Co-authored-by: Stephan Walter <[email protected]> Co-authored-by: DannyDaemonic <[email protected]> Co-authored-by: Erik Scholz <[email protected]> Co-authored-by: Georgi Gerganov <[email protected]> Co-authored-by: David Kennedy <[email protected]> Co-authored-by: Jason McCartney <[email protected]> Co-authored-by: Evan Jones <[email protected]> Co-authored-by: Maxime <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Zenix <[email protected]>
AlexiAlp
pushed a commit
to minghaop/llama.cpp
that referenced
this pull request
* examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]>
AlexiAlp
pushed a commit
to minghaop/llama.cpp
that referenced
this pull request
…oadcasting for ggml_mul (ggml-org#1483) * Broadcasting for ggml_mul * CUDA kernel for ggml_mul, norms in VRAM * GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * fixup! GPU weights not in RAM, direct loading with cuFile * define default model path once, sync path with readme (ggml-org#1366) * ~7% faster Q5_1 AVX2 code (ggml-org#1477) * convert.py: Support models which are stored in a single pytorch_model.bin (ggml-org#1469) * Support models in a single pytorch_model.bin * Remove spurious line with typo * benchmark-matmul: Print the average of the test results (ggml-org#1490) * Remove unused n_parts parameter (ggml-org#1509) * Fixes ggml-org#1511 lambda issue for w64devkit (mingw) (ggml-org#1513) * Fix for w64devkit and mingw * make kv_f16 the default for api users (ggml-org#1517) * minor : fix compile warnings * readme : adds WizardLM to the list of supported models (ggml-org#1485) * main : make reverse prompt option act as a stop token in non-interactive mode (ggml-org#1032) * Make reverse prompt option act as a stop token in non-interactive scenarios * Making requested review changes * Update gpt_params_parse and fix a merge error * Revert "Update gpt_params_parse and fix a merge error" This reverts commit 2bb2ff1. * Update gpt_params_parse and fix a merge error take 2 * examples : add persistent chat (ggml-org#1495) * examples : add persistent chat * examples : fix whitespace --------- Co-authored-by: Georgi Gerganov <[email protected]> * tests : add missing header * ggml : use F16 instead of F32 in Q4_0, Q4_1, Q8_0 (ggml-org#1508) * ggml : use F16 instead of F32 in Q4_0, Q4_1 and Q8_0 * llama : bump LLAMA_FILE_VERSION to 3 * cuda : update Q4 and Q8 dequantize kernels * ggml : fix AVX dot products * readme : update performance table + hot topics * ggml : fix scalar implementation of Q4_1 dot * llama : fix compile warnings in llama_set_state_data() * llama : fix name shadowing and C4146 (ggml-org#1526) * Fix name shadowing and C4146 * Fix if macros not using defined when required * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update llama-util.h Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Code style Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Georgi Gerganov <[email protected]> * Fix for mingw (ggml-org#1462) * llama : add llama_init_backend() API (close ggml-org#1527) * feature : add blis and other BLAS implementation support (ggml-org#1502) * feature: add blis support * feature: allow all BLA_VENDOR to be assigned in cmake arguments. align with whisper.cpp pr 927 * fix: version detection for BLA_SIZEOF_INTEGER, recover min version of cmake * Fix typo in INTEGER Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: Georgi Gerganov <[email protected]> * Revert "feature : add blis and other BLAS implementation support (ggml-org#1502)" This reverts commit 65c6ec3. * GPU weights not in RAM, direct loading with cuFile * llama : code style fixes + progress print fix * ggml : ggml_mul better broadcast support * cmake : workarounds for cufile when CMake version < 3.25 * gg rebase fixup * Loop in llama.cpp, fixed progress callback * Attempt clang-tidy fix * llama : fix vram size computation * Add forgotten fclose() --------- Co-authored-by: András Salamon <[email protected]> Co-authored-by: Ilya Kurdyukov <[email protected]> Co-authored-by: Tom Jobbins <[email protected]> Co-authored-by: rankaiyx <[email protected]> Co-authored-by: Stephan Walter <[email protected]> Co-authored-by: DannyDaemonic <[email protected]> Co-authored-by: Erik Scholz <[email protected]> Co-authored-by: Georgi Gerganov <[email protected]> Co-authored-by: David Kennedy <[email protected]> Co-authored-by: Jason McCartney <[email protected]> Co-authored-by: Evan Jones <[email protected]> Co-authored-by: Maxime <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Zenix <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ReviewersUh oh!
There was an error while loading. Please reload this page. AssigneesNo one assignedLabels
None yet
ProjectsMilestoneNo milestoneDevelopmentSuccessfully merging this pull request may close these issues. Uh oh!
There was an error while loading. Please reload this page. 2 participantsAdd this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later. | ||
| You are visitor 01402503 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |
