Python bindings for ggml
Python bindings for the ggml tensor library for machine learning.
⚠️ Neither this project norggmlcurrently guarantee backwards-compatibility, if you are using this library in other applications I strongly recommend pinning to specific releases in yourrequirements.txtfile.
Requirements
- Python 3.8+
- C compiler (gcc, clang, msvc, etc)
You can install ggml-python using pip:
pip install ggml-python
This will compile ggml using cmake which requires a c compiler installed on your system.
To build ggml with specific features (ie. OpenBLAS, GPU Support, etc) you can pass specific cmake options through the cmake.args pip install configuration setting. For example to install ggml-python with cuBLAS support you can run:
pip install --upgrade pip
pip install ggml-python --config-settings=cmake.args='-DGGML_CUDA=ON'| Option | Description | Default |
|---|---|---|
GGML_CUDA |
Enable cuBLAS support | OFF |
GGML_CLBLAST |
Enable CLBlast support | OFF |
GGML_OPENBLAS |
Enable OpenBLAS support | OFF |
GGML_METAL |
Enable Metal support | OFF |
GGML_RPC |
Enable RPC support | OFF |
import ggml import ctypes # Allocate a new context with 16 MB of memory params = ggml.ggml_init_params(mem_size=16 * 1024 * 1024, mem_buffer=None) ctx = ggml.ggml_init(params) # Instantiate tensors x = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1) a = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1) b = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1) # Use ggml operations to build a computational graph x2 = ggml.ggml_mul(ctx, x, x) f = ggml.ggml_add(ctx, ggml.ggml_mul(ctx, a, x2), b) gf = ggml.ggml_new_graph(ctx) ggml.ggml_build_forward_expand(gf, f) # Set the input values ggml.ggml_set_f32(x, 2.0) ggml.ggml_set_f32(a, 3.0) ggml.ggml_set_f32(b, 4.0) # Compute the graph ggml.ggml_graph_compute_with_ctx(ctx, gf, 1) # Get the output value output = ggml.ggml_get_f32_1d(f, 0) assert output == 16.0 # Free the context ggml.ggml_free(ctx)
If you are having trouble installing ggml-python or activating specific features please try to install it with the --verbose and --no-cache-dir flags to get more information about any issues:
pip install ggml-python --verbose --no-cache-dir --force-reinstall --upgrade
This project is licensed under the terms of the MIT license.