Building an E-Commerce Image Search Application with Marqo's Advanced Multi-Models

Overview

This blog explains how making a great e-commerce image search application can look hard, but it’s not with the sophisticated multimodal embeddings from Marqo. This blog will guide you through setting up a Marqo-powered image search engine, including indexing and user interface design. A searchable application that is complete and ready to use will be at your fingertips by the end of this blog.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Introduction

In the fast-paced world of e-commerce, high-quality product searches and recommendations represent that gold standard. Morqo has introduced two novel foundation models: Marqo-Ecommerce-B and Marqo-Ecommerce-L. State-of-the-art models are particularly optimized for e-commerce applications in multimodal capability by generating superior product embeddings from images and text.

With up to 88% performance over best-in-class solutions such as Amazon Titan’s Multimodal Embedding and 31% outperformance of the best open-source model (ViT-SO400M-14-SigLIP), Marqo-Ecommerce models are on a whole new level.

Their unmatched capacity to comprehend and rank products in realistic search scenarios enables businesses to create personalized, accurate, high-quality search and recommendation experiences.

Dive into this blog to discover how these models transform e-commerce and why they have become the go-to for the best product discovery and user engagement.

Benchmarking Results

Marqo-E-commerce-Hard

Text-to-Image Retrieval:
- Marqo-Ecommerce-L outperformed Amazon Titan Multimodal with a 43.7% improvement in MRR and 35.4% in Recall@10.
- Against ViT-SO400M-14-SigLIP achieved a 19% boost in MRR and 15% in Recall@10.
Category-to-Image Retrieval:
- Marqo-Ecommerce-L improved mAP by 88%, Precision@10 by 52%, and nDCG@10 by 49.3% over Amazon Titan Multimodal.
- It also outperformed ViT-SO400M-14-SigLIP with a 31.5% gain in mAP and 25.9% in nDCG@10.
Amazon Product Text-to-Image Retrieval:
- Achieved a 45% improvement in MRR, 43% in nDCG@10, and 36% in Recall@10 compared to Amazon Titan Multimodal.

Marqo-Ecommerce-Easy

For smaller datasets, Marqo models consistently excelled while including additional competitors such as Cohere-Embedding-v3 and GCP-Vertex. Performance gains mirrored those seen with larger datasets, emphasizing the models’ robustness across varying scales.

Step-by-Step Guide

Step 1: Creating a Marqo Index
The first step is creating an index in Marqo Cloud. This index will contain data for products, such as titles, categories, and images, to enable fast and accurate search results using Marqo’s advanced embeddings.

1.1 Initialize and Configure Marqo
Begin with initializing a Marqo client: preparing the environment and setting up an API key.

import marqo
import os
from dotenv import load_dotenv

# Load the environment variables
load_dotenv()

# Set up Marqo Client 
api_key = os.getenv("MARQO_API_KEY_Credentials")   
mq = marqo.Client("https://api.marqo.ai", api_key=api_key)

import marqo

import os

from dotenv import load_dotenv

# Load the environment variables

load_dotenv()

# Set up Marqo Client

api_key = os.getenv("MARQO_API_KEY_Credentials")

mq = marqo.Client("https://api.marqo.ai", api_key=api_key)

Alternatively, if you want to run Marqo’s model locally, use the following GitHub repository instead. For information on how to set up Marqo locally, see GitHub.

1.2 Index Settings- Definition

Example configuration for marqo-ecommerce-embeddings-L:

step1

1.3 Create the Index

Now, delete any existing index with the exact same name (to avoid unnecessary conflicts) and create a new one index.

index__name = "marqo-ecommerce-l1"

try:
    mq.index(index__name).delete()
except:
    pass  # Ignore if the index doesn’t exist

mq.create_index(index__name, settings_dict=settings)

index__name = "marqo-ecommerce-l1"

try:

mq.index(index__name).delete()

except:

pass # Ignore if the index doesn’t exist

mq.create_index(index__name, settings_dict=settings)

Step 2: Adding Documents to the Index

Now, populate the index with the data of products from a dataset, such as a file in CSV format containing categories, titles, and image URLs.
2.1 Load and Prepare Data

import pandas as pd

# Load dataset
path_to_data = "data/marqo-gs_10k.csv"
df = pd.read_csv(actual_path_to_data)

import pandas as pd

# Load dataset

path_to_data = "data/marqo-gs_10k.csv"

df = pd.read_csv(actual_path_to_data)

2.2 Format Data for Marqo

Convert the data into a Marqo-compatible format.

step2

2.3 Upload Documents in Batches

Upload the documents in chunks to optimize performance.

step2b

Step 3: Deploy on any desired platform

I am deploying the application to Hugging Face Spaces for demo purposes and broader access. Define the Marqo API key as a secret variable.

Once the testing is completed, make sure to clean the resources.

Step 4: Clean Up

To avoid unnecessary charges, delete the index when no longer needed.

mq.delete_index(index__name)

1	mq.delete_index(index__name)

Conclusion

Marqo-Ecommerce-B and Marqo-Ecommerce-L represent a leap forward in e-commerce AI. With unprecedented precision and recall, these models set a new standard for search and recommendation systems, empowering businesses to deliver exceptional customer experiences and drive revenue growth.

Drop a query if you have any questions regarding Marqo and we will get back to you quickly.

Empowering organizations to become ‘data driven’ enterprises with our Cloud experts.

Reduced infrastructure costs
Timely data-driven decisions

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is the first Indian Company to win the prestigious Microsoft Partner 2024 Award and is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, AWS GenAI Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, Amazon ECS Service Delivery Partner, AWS Glue Service Delivery Partner, Amazon Redshift Service Delivery Partner, AWS Control Tower Service Delivery Partner, AWS WAF Service Delivery Partner, Amazon CloudFront and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.

FAQs

1. What are the Marqo-Ecommerce-B and Marqo-Ecommerce-L models?

ANS: – These are advanced foundation models optimized for e-commerce applications. They generate high-quality multimodal product embeddings from images and text so that product search and recommendation could perform better than existing solutions.

2. How do these models outperform existing solutions?

ANS: – Marqo-Ecommerce models reach as much as 88% more efficiently than Amazon Titan’s Multimodal Embedding and 31% compared to the best open-source model: ViT-SO400M-14-SigLIP. They are especially fine-tuned for real-world e-commerce scenarios.

WRITTEN BY Aditya Kumar

Aditya Kumar works as a Research Associate at CloudThat. His expertise lies in Data Analytics. He is learning and gaining practical experience in AWS and Data Analytics. Aditya is also passionate about continuously expanding his skill set and knowledge to learn new skills. He is keen to learn new technology.