Unlock the full potential of your analytics data with Amazon S3 Tables

Voiced by Amazon Polly

Amazon Web Services (AWS) has announced Amazon S3 Tables, a new storage solution designed specifically for analytics workloads. Built to handle tabular data efficiently, such as daily transactions, sensor data, and ad impressions, S3 Tables offer superior query performance, higher transactions per second, and reduced storage costs—all while providing the operational ease of a fully managed service.

Transform Your Career with AWS Certifications

Advanced Skills
AWS Official Curriculum
10+ Hand-on Labs

Enroll Now

Key Highlights of Amazon S3 Tables:

Purpose-Built Storage for Analytics: Compared to self-managed tables in S3 general-purpose buckets, S3 Tables ensure seamless integration with analytics engines such as Amazon Athena, Amazon EMR, and Apache Spark, delivering up to 3x faster queries and up to 10x more transactions per second.
Built-In Apache Iceberg Support: S3 Tables leverage the Apache Iceberg format, a widely adopted table format for large-scale analytics.
Automated Table Optimization: Amazon S3 Tables continuously optimize storage through: Data Compaction, Snapshot Management and Unreferenced File Cleanup.
Access Management and Security: It use AWS Identity and Access Management (IAM) for fine-grained access control.
Seamless Integration with AWS Analytics Services: Amazon S3 Tables integrate with AWS services, enabling data analytics without complex configurations: Amazon Athena, AWS Glue, Amazon EMR, Amazon Redshift, Amazon QuickSight and Amazon Lake Formation.

Steps to Use S3 Tables for Analytics

Create S3 Table Bucket

Connecting to S3 table buckets with Spark on an Amazon EMR Iceberg cluster

aws emr create-cluster \

  --release-label emr-7.5.0 \

  --applications Name=Spark \

  --configurations file://configurations.json \

  --region us-east-1 \

  --name My_cluster_Iceberg_demo \

  --instance-type m5.xlarge \

  --instance-count 3 \

  --log-uri <a href="https://us-east-1.console.aws.amazon.com/s3/buckets/iceberg-logs-8122024/j-2U0NISO7CE15Q/?region=us-east-1">s3://iceberg-logs-8122024</a>/ \

  --service-role EMR_DefaultRole \

  --ec2-attributes InstanceProfile=EMR_EC2_DefaultRole

aws emr create-cluster \

--release-label emr-7.5.0 \

--applications Name=Spark \

--configurations file://configurations.json \

--region us-east-1 \

--name My_cluster_Iceberg_demo \

--instance-type m5.xlarge \

--instance-count 3 \

--log-uri <a href="https://us-east-1.console.aws.amazon.com/s3/buckets/iceberg-logs-8122024/j-2U0NISO7CE15Q/?region=us-east-1">s3://iceberg-logs-8122024</a>/ \

--service-role EMR_DefaultRole \

--ec2-attributes InstanceProfile=EMR_EC2_DefaultRole

Configuration.json is as follows:

[ {

    "Classification": "iceberg-defaults",

    "Properties": { "iceberg.enabled": "true"}

  } ]

[ {

"Classification": "iceberg-defaults",

"Properties": { "iceberg.enabled": "true"}

} ]

Once the EMR cluster has been created then connect with the primary node of EMR
Then next step is to initialize the Spark Session for Iceberg that connects your tables

pyspark \

 --packages 'software.amazon.s3tables:s3-tables-catalog-for-iceberg:0.1.0,org.apache.iceberg:iceberg-spark-runtime-3.4_2.12:1.6.0'

pyspark \

--packages 'software.amazon.s3tables:s3-tables-catalog-for-iceberg:0.1.0,org.apache.iceberg:iceberg-spark-runtime-3.4_2.12:1.6.0'

from pyspark.sql import SparkSession

spark = SparkSession.builder.appName("iceberg_lab") \

    .config("spark.sql.catalog.s3tablesbucket", "org.apache.iceberg.spark.SparkCatalog") \

    .config("spark.sql.catalog.s3tablesbucket.client.region", "us-east-1") \

    .config("spark.sql.catalog.defaultCatalog", "s3tablesbucket") \

    .config("spark.sql.catalog.s3tablesbucket.warehouse", "arn:aws:s3tables:us-east-1:862820731644:bucket/demo-s3-iceberg-table") \

    .config("spark.sql.catalog.s3tablesbucket.catalog-impl", "software.amazon.s3tables.iceberg.S3TablesCatalog") \

    .getOrCreate()

from pyspark.sql import SparkSession

spark = SparkSession.builder.appName("iceberg_lab") \

.config("spark.sql.catalog.s3tablesbucket", "org.apache.iceberg.spark.SparkCatalog") \

.config("spark.sql.catalog.s3tablesbucket.client.region", "us-east-1") \

.config("spark.sql.catalog.defaultCatalog", "s3tablesbucket") \

.config("spark.sql.catalog.s3tablesbucket.warehouse", "arn:aws:s3tables:us-east-1:862820731644:bucket/demo-s3-iceberg-table") \

.config("spark.sql.catalog.s3tablesbucket.catalog-impl", "software.amazon.s3tables.iceberg.S3TablesCatalog") \

.getOrCreate()

Now you can work with database using Spark SQL

spark.sql("CREATE NAMESPACE IF NOT EXISTS demo.CloudThat_db")

spark.sql("""

CREATE TABLE IF NOT EXISTS demo.CloudThat_db.orders (

  order_id BIGINT,

  customer_id BIGINT,

  order_date DATE,

  total_amount DOUBLE

) USING iceberg

"""

)

spark.sql("CREATE NAMESPACE IF NOT EXISTS demo.CloudThat_db")

spark.sql("""

CREATE TABLE IF NOT EXISTS demo.CloudThat_db.orders (

order_id BIGINT,

customer_id BIGINT,

order_date DATE,

total_amount DOUBLE

) USING iceberg

"""

)

Conclusion

Amazon S3 Tables revolutionize data storage for analytics by offering a purpose-built, fully managed solution that seamlessly integrates with popular AWS analytics services. With features like built-in Apache Iceberg support, automated table optimization, and robust security through AWS IAM, S3 Tables deliver enhanced query performance, scalability, and cost efficiency. By simplifying complex configurations and enabling faster data processing, Amazon S3 Tables empower businesses to unlock deeper insights from their analytics workloads with ease and reliability.

Earn Multiple AWS Certifications for the Price of Two

AWS Authorized Instructor led Sessions
AWS Official Curriculum

Get Started Now

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is the first Indian Company to win the prestigious Microsoft Partner 2024 Award and is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, AWS GenAI Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, Amazon ECS Service Delivery Partner, AWS Glue Service Delivery Partner, Amazon Redshift Service Delivery Partner, AWS Control Tower Service Delivery Partner, AWS WAF Service Delivery Partner and many more.