GeezerVisonThe World Wide Web, restored to 800 pixels of glory. |
| Home|About|Gaming cabinet|Archives|How it works|Starter sites |
GeezerVison is showing github.com — Build software better, together
#
model-servingHere are 842 public repositories matching this topic...Language: All
Filter by language
Sort: Most stars
Sort options
A high-throughput and memory-efficient inference and serving engine for LLMs
amd cuda inference pytorch transformer openai moe llama gpt model-serving tpu kimi blackwell llm llm-serving qwen deepseek deepseek-v3 qwen3 gpt-oss
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
python machine-learning deep-learning model-serving multimodal mlops ml-engineering ai-inference llm generative-ai llmops llm-serving model-inference-service llm-inference inference-platform
Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts)
python workflow machine-learning reinforcement-learning scale mcp kubernetes-cluster agents model-serving model-training mlops ai-runtime sovereign-infrastructure sovereign-ai agent-runtime
A framework for efficient model inference with omni-modality models
inference pytorch transformer image-generation diffusion model-serving multimodal video-generation audio-generation world-model
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
kubernetes machine-learning tensorflow cncf pytorch artificial-intelligence xgboost k8s service-mesh hacktoberfest istio model-serving kubeflow mlops knative model-interpretability kserve vllm genai llm-inference
Open-Source Personal Cloud OS for Always-On Agents
kubernetes self-hosted operating-system cloud-native data-privacy homelab personal-cloud ai-agents model-serving edge-ai local-ai agentic-ai
🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice. ⚡️ System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI). 🍻 OSDI, NSDI, SIGCOMM, SoCC, MLSys, etc. 🗃️ Llama3, Mistral, etc. 🧑💻 Video Tutorials.
In this repository, I will share some useful notes and references about deploying deep learning-based models in production.
react python angularjs flask c-plus-plus tutorial deep-neural-networks deep-learning mxnet rest-api production keras pytorch tensorflow-models tesnorflow model-serving caffe2 serving serving-pytorch-models convert-pytorch-models
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.
machine-learning deep-learning inference-engine model-deployment model-serving distributed-training federated-learning mlops edge-ai ai-agent on-device-training
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
OpenLake is a high performance storage engine for efficient LLM inference and GPU Training
Community maintained hardware plugin for vLLM on Huawei Ascend
High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
🏕️ Reproducible development environment for humans and agents
agent docker developer-tools development-environment hacktoberfest codex model-serving buildkit mlops mlops-workflow llmops code-agent
AICI: Prompts as (Wasm) Programs
rust ai wasm inference transformer language-model model-serving wasmtime llm llmops llm-serving llm-inference llm-framework
High Performance Serving Runtime For Interactive Super Intelligence System
text-to-speech streaming interactive cuda inference pytorch tts speech-recognition music-generation asr model-serving multimodal openai-api audio-generation sglang distributed-inference
MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.
python kubernetes workflow data-science machine-learning data-engineering model-serving mlops experiment-tracking mlops-workflow
Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app, Docker, MCP server, public bench.
rust self-hosted calibration typesafe decision-model inference-server model-serving laya systemone onnxruntime local-inference llama-cpp ollama mcp-server decision-models jev system-one typed-decisions jev-alternative jev-compatible
An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.
kubernetes devops opensource ai code tensorflow sklearn models ml pytorch datasets devops-tools hacktoberfest kubernetes-deployment model-serving platform-engineering mlops model-interpretability gguf mlops-tools
Add this topic to your repoTo associate your repository with the model-serving topic, visit your repo's landing page and select "manage topics." |
| You are visitor 01402503 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |