Managed Service for Apache Spark

GCPAnalytics

Managed Apache Spark and Hadoop (formerly Dataproc and Serverless for Apache Spark), offered as per-second-billed clusters on Compute Engine or as serverless Spark batches and sessions, with Flink, Trino and other open-source components.

Jurisdictional exposure

Provider HQ
USMountain View, USA

Subject to CLOUD Act, FISA-702, DPF

Region locations
APACCNEEAEUUKUSOther44 regions across 7 jurisdictions
Sovereign option
Yes — 1 sovereign-flagged region available

Attributes

Auto Scaling
Yes
Spark Support
Yes
Hadoop Support
Yes

Sub-services (3)

Dataproc Clusters

Managed Spark and Hadoop cluster provisioning

Dataproc Serverless

Serverless Spark for batch workloads

Dataproc Metastore

Managed Hive Metastore for metadata management

Compliance & Certifications

This service is attested for the following frameworks. Always verify with the provider before relying on a specific compliance posture.

Where this runs

44 regions
29 countries
1sovereign
Sovereign regions (1)
  • T-Systems Sovereign Cloud · FrankfurtT-Systems Sovereign Cloud powered by Google Cloud
Commercial regions (43)

Europe (13)

  • Belgium
  • Finland
  • Paris
  • Berlin
  • Frankfurt
  • Milan
  • Turin
  • Netherlands
  • Warsaw
  • Madrid
  • Stockholm
  • Zurich
  • London

North America (12)

  • Montréal
  • Toronto
  • Querétaro
  • Northern Virginia
  • Columbus
  • Iowa
  • Dallas
  • Las Vegas
  • Los Angeles
  • South Carolina
  • Salt Lake City
  • Oregon

South America (2)

  • São Paulo
  • Santiago

Asia (10)

  • Hong Kong
  • Delhi
  • Mumbai
  • Jakarta
  • Osaka
  • Tokyo
  • Singapore
  • Seoul
  • Taiwan
  • Bangkok

Oceania (2)

  • Melbourne
  • Sydney

Middle East (3)

  • Tel Aviv
  • Doha
  • Dammam

Africa (1)

  • Johannesburg

Tags

Equivalent services on other platforms

Alibaba MaxComputeAlibaba

Exabyte-scale fully managed data warehouse (formerly ODPS) optimised for batch analytics on structured and semi-structured data, with SQL, MapReduce, and Spark computation engines, columnar storage, and tight integration with DataWorks

Amazon AthenaAWS

Serverless interactive query service that runs ANSI SQL directly on S3 data lakes with per-query pricing, Apache Iceberg and Hudi table formats, and Glue catalog integration

Amazon EMRAWS

Managed big-data platform for running Apache Spark, Hive, Presto, Flink, Trino, and HBase across EC2, EKS, and fully serverless deployments with up to 5x faster Spark runtime and Graviton price-performance

Azure Synapse AnalyticsAzure

Unified analytics service combining data warehousing and big data processing with dedicated and serverless SQL pools, Apache Spark, Data Integration pipelines, and Power BI embedded

Azure DatabricksAzure

First-party Microsoft-co-engineered deployment of the Databricks Lakehouse with native Entra ID sign-in, private networking, serverless SQL warehouses, Unity Catalog governance, Delta Lake storage, and Azure Portal billing — the sanctioned way to run Databricks inside an Azure subscription

Azure HDInsightAzure

Managed open-source analytics clusters for Hadoop, Apache Spark, Apache Hive LLAP, Apache Kafka, and Apache HBase with enterprise security via Enterprise Security Package, autoscale, and integration with ADLS Gen2 — used mainly for migrating existing OSS big-data estates into Azure

Databricks Data + AI PlatformDatabricks

Unified data lakehouse combining the reliability of data warehouses with the scale of data lakes, built on Delta Lake with ACID transactions, schema enforcement, and time travel

Huawei MapReduce ServiceHuawei

Fully managed big-data platform running Apache Spark, Hive, HBase, Flink, Hadoop, and Kudu clusters with autoscaling, Kerberos security, and integration with OBS for compute-storage separation

Hadoop EcoKakao

Managed Hadoop ecosystem clusters (Hadoop, Spark, Hive, HBase and more) for big-data processing

OCI Big Data ServiceOracle

Managed Hadoop and Spark clusters built on Oracle Distribution including Apache Hadoop (ODH), with Hive, HBase, Spark, Trino and Ambari management, highly available master/worker/edge nodes and OCI integrations.

OCI Data FlowOracle

Serverless managed Apache Spark — submit a Spark job, OCI runs it on transient infrastructure, returns the result. No cluster to manage; pay per OCPU-second of execution.

OVHcloud Data PlatformOVHcloud

Unified data platform, based on ForePaaS, combining connectors, an ETL/ELT processing engine, an Apache Iceberg lakehouse, a Trino-based analytics and dashboard manager, and serverless app services. Compute is billed per vCPU-hour and data scanned; there is a free tier.

Tencent Elastic MapReduceTencent

Managed big-data platform running Apache Spark, Hadoop, Hive, HBase, Flink, Presto, and ClickHouse with autoscaling, Kerberos authentication, integration with COS for compute-storage separation, and Jupyter notebooks for interactive analysis

Pricing

Pricing model:pay-as-you-go