Managed Inference and Agents

HerokuAI & ML

Heroku add-on giving OpenAI-style API access to chat, embedding, rerank and image models hosted on Amazon Bedrock, with tool execution on dynos and MCP server support; US and EU model sets

Jurisdictional exposure

Provider HQ
USSan Francisco, USA

Subject to CLOUD Act, FISA-702, DPF

Region locations
APACEUUKUSOther12 regions across 5 jurisdictions
Sovereign option
No sovereign-flagged regions in the catalogue for this service.

Sub-services (4)

Chat Completions

OpenAI-compatible endpoint for Claude, Amazon Nova and gpt-oss models

Embeddings and Rerank

Cohere embedding models plus Cohere and Amazon rerank models

Heroku Tools

Built-in and custom tools, such as code execution, run on dynos

MCP Servers

Deploy MCP servers on Heroku and expose them to models as tools

Compliance & Certifications

This service is attested for the following frameworks. Always verify with the provider before relying on a specific compliance posture.

Where this runs

12 regions
9 countries
Commercial regions (12)

Europe (4)

  • Private Space — Frankfurt
  • Common Runtime — EU (Ireland)
  • Private Space — Dublin
  • Private Space — London

North America (4)

  • Private Space — Montreal
  • Common Runtime — US (Virginia)
  • Private Space — Virginia
  • Private Space — Oregon

Asia (3)

  • Private Space — Mumbai
  • Private Space — Tokyo
  • Private Space — Singapore

Oceania (1)

  • Private Space — Sydney

Tags

Equivalent services on other platforms

Alibaba Qwen (Tongyi Qianwen)Alibaba

Alibaba's Qwen model family, from proprietary API models (Qwen-Max, Qwen-Plus, Qwen-Flash) to open-weight Qwen releases, served through Alibaba Cloud Model Studio's OpenAI-compatible API for chat, vision, audio, coding, embeddings and function calling

Aruba AI ModelsAruba

OpenAI-compatible API to open-weight models (e.g. gpt-oss-120b, Llama 3.1 8B, Mistral Small 3.2, Qwen3 Coder, bge-m3 embeddings, Qwen image models) hosted in Aruba's Italian data centres, billed per token or per image.

Amazon BedrockAWS

Build generative AI applications with foundation models from Anthropic (Claude Opus 4.7 from April 2026), Cohere, Meta, Mistral, Stability AI, TwelveLabs (video understanding), and Amazon's own Nova family — accessed via a single API with fine-tuning, knowledge bases, agents, and a model marketplace for discovery and easy onboarding

Amazon NovaAWS

AWS-built foundation model family covering text (Micro, Lite, Pro, Premier), image generation (Canvas), and video generation (Reel) — accessed through the Bedrock runtime with tight pricing and low-latency streaming, launched at re:Invent 2024

Amazon Bedrock AgentCoreAWS

Production runtime for AI agents — managed memory, identity, gateway, observability, and tool integration so teams can ship agentic workflows on top of any framework (Strands Agents, LangGraph, CrewAI, vendor-direct) without rebuilding the operational substrate

Amazon Nova ActAWS

Foundation-model service for browser-based agents — purpose-built model and SDK for AI agents that automate form fill, search, booking, QA testing, and other web-UI workflows, distinct from the general-purpose Nova family by being action-oriented rather than chat-oriented

Amazon QuickAWS

Agentic AI workspace for business users combining chat agents over company data, deep research, Flows and Automate for workflow automation, Spaces for shared knowledge, and Quick Sight BI. AWS's successor to Amazon Q Business.

Azure OpenAI ServiceAzure

Enterprise access to OpenAI models (GPT-5 family, reasoning, image, audio and embedding models) through Microsoft Foundry, with Entra ID auth, private networking, global, data-zone or regional deployments, and pay-as-you-go or provisioned throughput.

Microsoft FoundryAzure

Platform for building, deploying and governing AI apps and agents (formerly Azure AI Studio and Azure AI Foundry): model catalogue spanning OpenAI and partner models, Foundry Agent Service with MCP support, Foundry IQ knowledge, evaluation, tracing and a control plane.

relaxAICivo

UK-hosted AI assistant and OpenAI-compatible API from Civo, with chat subscriptions per user and per-token API pricing; data processed in UK data centres

Cloudflare Workers AICloudflare

Serverless GPU-backed AI inference at the edge running a catalogue of open-source text, image, speech, and embedding models (Llama, Mistral, Stable Diffusion, Whisper, BGE) with pricing per Neuron (Workers AI's billing unit) and direct binding from Workers code

Cloud Temple LLM-as-a-ServiceCloud Temple

Hosted large-language-model inference endpoints on SecNumCloud-qualified GPUs. Positioned as the French sovereign alternative to hosted OpenAI / Anthropic APIs for regulated workloads — health data (HDS-qualified), public-sector, defence-adjacent

Mosaic AIDatabricks

Databricks' AI platform (built on the 2023 MosaicML acquisition) for building, fine-tuning, serving, evaluating and governing ML models, foundation models and AI agents on the lakehouse, including Model Serving, AI Search and Agent Bricks.

Databricks Model ServingDatabricks

Serverless real-time endpoints for custom ML models, agents and hosted foundation models (pay-per-token or provisioned throughput), with autoscaling, gateway guardrails and Unity Catalog governance.

InferenceDigitalOcean

Managed model inference with a catalogue of DigitalOcean-hosted and third-party models, serverless and dedicated deployments, and request routing across providers

Exoscale Dedicated InferenceExoscale

Concrete AI service that deploys public, gated or private Hugging Face models on dedicated NVIDIA GPUs in European data centres as OpenAI-compatible API endpoints, billed per second of GPU time plus Object Storage, with scale-to-zero

Gcore Everywhere InferenceGcore

Serverless AI inference platform that deploys open-source or custom containerised models on NVIDIA GPUs in chosen regions, with smart routing across 210+ PoPs to the nearest endpoint and per-second GPU billing

Gemini Enterprise Agent PlatformGCP

Google Cloud's AI development platform (formerly Vertex AI): Model Garden access to Gemini and 200+ models, Agent Studio and ADK for building agents, Agent Runtime, managed training and tuning on TPUs/GPUs, pipelines, model registry and Vector Search.

Gemini APIGCP

Direct API access to Google's most capable multimodal AI models with text, image, audio, and video understanding, long context windows, and function calling support

Gemini EnterpriseGCP

Enterprise AI-agent platform combining Gemini reasoning with unified search across Workspace, Drive, ServiceNow, Jira, Salesforce, and custom sources — agents can act on tools + APIs on the user's behalf, with governance controls for what each agent may access

ModelArtsHuawei

End-to-end AI development platform with AutoML, data labelling, distributed training on Ascend and GPU clusters, and one-click deployment to cloud or edge

Pangu ModelsHuawei

Huawei's Pangu family of large models for NLP, multimodal, prediction and scientific computing, with model development, fine-tuning and deployment on Ascend infrastructure

IBM watsonx.aiIBM

Enterprise AI studio for training, validating, tuning, and deploying foundation models and traditional ML models, with IBM's Granite model family, Hugging Face integration, prompt lab, synthetic data generation, and governance via watsonx.governance

IBM watsonx OrchestrateIBM

Agent management platform for building, deploying, orchestrating and governing AI agents and assistants from one control plane; the upgrade path for watsonx Assistant

Infomaniak AI ServicesInfomaniak

OpenAI-compatible API to open-weight models hosted in Infomaniak's Swiss data centres: LLMs (e.g. Qwen 3.5, Mistral Small 4, Kimi K2.6, Apertus, Gemma 4), embeddings, rerankers, Whisper transcription and image generation; billed per token or per minute.

IONOS AI Model HubIONOS

Serverless inference API for a curated catalogue of open-weight models (e.g. Llama, Mistral, Qwen, gpt-oss, FLUX image and BGE embedding models) hosted in Berlin, with an OpenAI-compatible API, built-in vector stores for RAG, and per-token or per-image billing.

Advanced AI SpaceKakao

Deploys built-in catalogue or custom AI models to dedicated GPU-backed inference endpoints called via API key; available to pre-approved customers

CLOVA StudioNaver

Naver's HyperCLOVA X foundation-model platform for Korean-language LLM workloads — chat completion, embeddings, function calling, RAG over Korean text with strong native-language performance

Nscale Serverless InferenceNscale

Self-serve, prepaid serverless inference API for open-weight chat, vision, embedding and image-generation models on Nscale GPU infrastructure, with OpenAI-compatible endpoints and token-based pay-as-you-go pricing.

NumSpot Mistral AI PlatformNumSpot

Managed generative AI platform hosting Mistral AI models, operated end to end in France on NumSpot's sovereign cloud; access on request through sales rather than self-service

OCI Generative AIOracle

OCI's managed generative AI platform: on-demand and dedicated-cluster access to Cohere, Meta Llama, Google Gemini, OpenAI gpt-oss and xAI Grok models, fine-tuning, OpenAI-compatible APIs, and Enterprise AI features for building and deploying agents.

Outscale AI StudioOutscale

AI Studio (optionally with Le Chat and Vibe agent tooling) delivered on dedicated 8x H200 GPU capacity in the SecNumCloud cloudgouv-eu-west-1 region, giving access to a library of sovereign LLMs; sold on quotation

Outscale LLM as a ServiceOutscale

Dedicated Mistral AI model deployments (Ministral 8B, Mistral Small, Medium 3, Codestral, Voxtral, Embed) on reserved L40, H100 or H200 GPUs, sold as fixed monthly bundles in the SecNumCloud cloudgouv-eu-west-1 region

OVHcloud AI EndpointsOVHcloud

Serverless API access to 20+ open-weight models (LLMs, speech-to-text and text-to-speech, document and image analysis) through an OpenAI-compatible API. Billed per input and output token and hosted in the EU, with a free sandbox and a Batch mode in beta.

Einstein AISalesforce

AI layer integrated across Salesforce products: predictive lead scoring, opportunity scoring, generative AI for emails and summaries

Salesforce AgentforceSalesforce

Salesforce platform for building, deploying and governing autonomous AI agents grounded in CRM data — Atlas Reasoning Engine, Agent Builder, Agent Script, Agentforce Voice, MCP support, multi-agent orchestration and Data 360 retrieval

Scaleway Generative APIsScaleway

OpenAI-compatible inference for open-weight models (e.g. gpt-oss, Mistral, Qwen) hosted in Paris: Serverless billed per million tokens with 1M free tokens, plus Dedicated Deployment (formerly Managed Inference) billed hourly

Snowflake CortexSnowflake

Fully managed AI and ML service offering hosted LLMs, vector search, and ML functions inside Snowflake SQL

Snowflake CoWorkSnowflake

Ready-to-use agentic application (formerly Snowflake Intelligence) that lets business users ask natural-language questions across structured and unstructured data governed in Snowflake and act through configured agents.

STACKIT AI Model ServingSTACKIT

Managed inference API for shared open-weight chat and embedding models (e.g. gpt-oss, Qwen3, Gemma 4, Llama 3.3 70B), billed per million input and output tokens and hosted in STACKIT's EU01 region; launched May 2025

Tencent HY (Hunyuan)Tencent

Tencent's in-house Hunyuan (Tencent HY) large language models, including Hy3 (256K context), Hy4 preview (1M context) and Hy-MT2 translation models, served through Tencent Cloud's LLM Service TokenHub API alongside third-party models

Vultr Serverless InferenceVultr

Serverless platform for deploying and scaling AI model inference without managing GPU infrastructure

Pricing

Pricing model:per-token