A comprehensive toolkit for deploying production-ready Generative AI infrastructure on Amazon EKS. Includes pre-configured components for: π AI Gateway (LiteLLM) π€ LLM Serving (vLLM, SGLang, Ollama) π Vector Databases, π Embedding Models (TEI) π Observability (Langfuse, Phoenix) etc. Fast-track your GenAI deployment with Kubernetes
aws-samples/
sample-genai-on-eks-starter-kit
94
High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE. Includes VS Code extension, TypeScript/Python SDKs, and Docker deployment.
8.4k
cheshire-cat-ai/
core
cheshire-cat-ai/coreFrameworks & SDKs
AI agent microservice
3.1k
portel-dev/
ncp
portel-dev/ncpTools
Natural Context Provider (NCP). Your MCPs, supercharged. Find any tool instantly, load on demand, run on schedule, ready for any client. Smart loading saves tokens and energy.
98
hanyeol/
model-compose
hanyeol/model-composeFrameworks & SDKs
Deploy production-ready AI services in minutes. One YAML file for agents, RAG pipelines, and MCP servers β run anywhere. Inspired by docker-compose.
88