Skip to content

Latest commit

 

History

History
100 lines (70 loc) · 3.46 KB

README.md

File metadata and controls

100 lines (70 loc) · 3.46 KB
Raw

Python bindings for ggml

Documentation Status Tests PyPI PyPI - Python Version PyPI - License PyPI - Downloads

Python bindings for the ggml tensor library for machine learning.

⚠️ Neither this project nor ggml currently guarantee backwards-compatibility, if you are using this library in other applications I strongly recommend pinning to specific releases in your requirements.txt file.

Documentation

Installation

Requirements

  • Python 3.8+
  • C compiler (gcc, clang, msvc, etc)

You can install ggml-python using pip:

pip install ggml-python

This will compile ggml using cmake which requires a c compiler installed on your system. To build ggml with specific features (ie. OpenBLAS, GPU Support, etc) you can pass specific cmake options through the cmake.args pip install configuration setting. For example to install ggml-python with cuBLAS support you can run:

pip install --upgrade pip
pip install ggml-python --config-settings=cmake.args='-DGGML_CUDA=ON'

Options

Option Description Default
GGML_CUDA Enable cuBLAS support OFF
GGML_CLBLAST Enable CLBlast support OFF
GGML_OPENBLAS Enable OpenBLAS support OFF
GGML_METAL Enable Metal support OFF
GGML_RPC Enable RPC support OFF

Usage

import ggml
import ctypes

# Allocate a new context with 16 MB of memory
params = ggml.ggml_init_params(mem_size=16 * 1024 * 1024, mem_buffer=None)
ctx = ggml.ggml_init(params)

# Instantiate tensors
x = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
a = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
b = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)

# Use ggml operations to build a computational graph
x2 = ggml.ggml_mul(ctx, x, x)
f = ggml.ggml_add(ctx, ggml.ggml_mul(ctx, a, x2), b)

gf = ggml.ggml_new_graph(ctx)
ggml.ggml_build_forward_expand(gf, f)

# Set the input values
ggml.ggml_set_f32(x, 2.0)
ggml.ggml_set_f32(a, 3.0)
ggml.ggml_set_f32(b, 4.0)

# Compute the graph
ggml.ggml_graph_compute_with_ctx(ctx, gf, 1)

# Get the output value
output = ggml.ggml_get_f32_1d(f, 0)
assert output == 16.0

# Free the context
ggml.ggml_free(ctx)

Troubleshooting

If you are having trouble installing ggml-python or activating specific features please try to install it with the --verbose and --no-cache-dir flags to get more information about any issues:

pip install ggml-python --verbose --no-cache-dir --force-reinstall --upgrade

License

This project is licensed under the terms of the MIT license.