10. より賢い検索: フィルタ、しきい値、MMR

# Refund Policy
 
**Effective Date**: January 1, 2026
 
## Standard Returns
 
All physical products may be returned within 30 days of purchase for a full refund.
The original receipt or order confirmation email is required. Items must be in their
original packaging and unused condition.
 
After 30 days, returns are accepted for store credit only. Store credit does not expire.
 
## Digital Products
 
Digital products (software licenses, e-books, online courses) are non-refundable
once the download or access link has been activated. If you experience technical
issues preventing access, contact support within 7 days for a replacement or refund.
 
## Defective Items
 
Defective items may be returned at any time for a full refund or replacement.
Please include a description of the defect. Shipping costs for defective returns
are covered by the company.
 
## Subscription Services
 
Monthly subscriptions may be cancelled at any time. Refunds are prorated based on
the remaining days in the billing cycle. Annual subscriptions may be refunded in full
within the first 14 days. After 14 days, no refund is available but access continues until the end of the billing period.

# Shipping Information
 
## Domestic Shipping
 
Standard shipping (5-7 business days): Free on orders over $50, otherwise $5.99.
Express shipping (2-3 business days): $12.99.
Overnight shipping (next business day): $24.99.
 
## International Shipping
 
International orders are shipped via tracked airmail. Delivery times vary by
destination, typically 10-21 business days. International shipping costs are
calculated at checkout based on weight and destination.
 
Customs duties and import taxes are the responsibility of the buyer and are not included in the shipping cost.
 
## Order Tracking
 
All orders include a tracking number sent via email within 24 hours of shipment.
Track your order through the tracking link in your email or through the carrier's website.
 
## Lost or Damaged Packages
 
If your package is lost or arrives damaged, contact support within 48 hours.
We will ship a replacement at no additional cost. For damaged items, please
provide photos of the damage and packaging.

# Employee Store Guide
 
## Eligibility and Benefits
 
Employees can purchase company products at a 30% discount through the internal employee store.
The monthly purchase limit is $500, and payment can be made via payroll deduction or benefit points.
 
## Ordering and Shipping
 
Employee store orders are placed through the internal portal, and delivery is only available to the company address.
Orders are delivered within 3-5 business days, and shipping is free.
 
## Refund Policy
 
Refunds are available within 7 days of purchase for unopened items only.
Cash refunds are not available; refunds are credited as benefit points.
After opening, only exchanges are allowed, limited to one exchange per identical product.
 
## Contact
 
For employee store inquiries, please contact HR at hr@acme.com.

# ingest.py — 第9章と同じインジェストパイプライン
from langchain_community.document_loaders import DirectoryLoader, TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma
 
loader = DirectoryLoader(
    "data/docs/", glob="**/*.md",
    loader_cls=TextLoader, loader_kwargs={"encoding": "utf-8"},
)
documents = loader.load()
print(f"Loaded {len(documents)} documents")
 
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=200,
    chunk_overlap=80,
    separators=["\n## ", "\n\n", "\n", " ", ""],
)
chunks = text_splitter.split_documents(documents)
print(f"Created {len(chunks)} chunks")
 
embedding_model = OpenAIEmbeddings(model="text-embedding-3-small")
 
vector_store = Chroma.from_documents(
    documents=chunks,
    embedding=embedding_model,
    persist_directory="data/chroma_db",
    collection_name="company_docs",
)
print(f"Stored {len(chunks)} chunks in ChromaDB")

Loaded 3 documents
Created 12 chunks
Stored 12 chunks in ChromaDB

from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma
 
embedding_model = OpenAIEmbeddings(model="text-embedding-3-small")
vector_store = Chroma(
    persist_directory="data/chroma_db",
    collection_name="company_docs",
    embedding_function=embedding_model,
)
 
results = vector_store.similarity_search("What are the refund conditions?", k=3)
 
for i, doc in enumerate(results):
    source = doc.metadata["source"]
    print(f"Result {i+1} [{source}] {doc.page_content[:80]}...")

Result 1 [data/docs/refund_policy.md] ## Standard Returns
All physical products may be returned within 30 days of pur...
 
Result 2 [data/docs/employee_store.md] ## Refund Policy
Refunds are available within 7 days of purchase for unopened i...
 
Result 3 [data/docs/refund_policy.md] ## Subscription Services
Monthly subscriptions may be cancelled at any time. Re...

results = vector_store.similarity_search_with_score(
    "What is the hiring process at this company?", k=3
)
 
for doc, score in results:
    source = doc.metadata["source"]
    print(f"[dist={score:.4f}] [{source}] {doc.page_content[:80]}...")

[dist=1.4648] [data/docs/employee_store.md] ## Refund Policy
Refunds are available within 7 days of purchase for unopened i...
 
[dist=1.4699] [data/docs/employee_store.md] ## Eligibility and Benefits
Employees can purchase company products at a 30% di...
 
[dist=1.4743] [data/docs/refund_policy.md] ## Subscription Services
Monthly subscriptions may be cancelled at any time. Re...

from rag_chain import build_rag_chain  # 第9章のRAGチェーン
 
chain = build_rag_chain()
answer = chain.invoke("What is the hiring process at this company?")
print(answer)

I don't have enough information to answer that question.

# ingest_with_metadata.py
from langchain_community.document_loaders import DirectoryLoader, TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma
 
# ステップ1: ドキュメントを読み込む (10.1と同じ)
loader = DirectoryLoader(
    "data/docs/", glob="**/*.md",
    loader_cls=TextLoader, loader_kwargs={"encoding": "utf-8"},
)
documents = loader.load()
print(f"Loaded {len(documents)} documents")
 
# ステップ2: [新規] ファイル名に基づいてカテゴリのメタデータを割り当てる
CATEGORY_MAP = {
    "refund_policy.md": "customer",
    "shipping_info.md": "customer",
    "employee_store.md": "employee",
}
for doc in documents:
    filename = doc.metadata["source"].split("/")[-1]
    doc.metadata["category"] = CATEGORY_MAP.get(filename, "unknown")
 
# ステップ3: チャンクに分割する (10.1と同じ)
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=200,
    chunk_overlap=80,
    separators=["\n## ", "\n\n", "\n", " ", ""],
)
chunks = text_splitter.split_documents(documents)
print(f"Created {len(chunks)} chunks")
 
# ステップ4: ベクトルストアを構築する (10.1と同じ)
embedding_model = OpenAIEmbeddings(model="text-embedding-3-small")
 
vector_store = Chroma.from_documents(
    documents=chunks,
    embedding=embedding_model,
    persist_directory="data/chroma_db",
    collection_name="company_docs",
)
print(f"Stored {len(chunks)} chunks in ChromaDB")

Loaded 3 documents
Created 12 chunks
Stored 12 chunks in ChromaDB

results = vector_store.similarity_search(
    "What are the refund conditions?",
    k=3,
    filter={"category": "customer"},
)
 
for i, doc in enumerate(results):
    source = doc.metadata["source"]
    category = doc.metadata["category"]
    print(f"Result {i+1} [{category}] [{source}] {doc.page_content[:80]}...")

Result 1 [customer] [data/docs/refund_policy.md] ## Standard Returns
All physical products may be returned within 30 days of pur...
 
Result 2 [customer] [data/docs/refund_policy.md] ## Subscription Services
Monthly subscriptions may be cancelled at any time. Re...
 
Result 3 [customer] [data/docs/refund_policy.md] ## Digital Products
Digital products (software licenses, e-books, online course...

# $and: すべての条件を満たすチャンクのみ
filter={
    "$and": [
        {"category": "customer"},
        {"source": "data/docs/refund_policy.md"},
    ]
}
 
# $or: いずれかの条件を満たすチャンク
filter={
    "$or": [
        {"source": "data/docs/refund_policy.md"},
        {"source": "data/docs/shipping_info.md"},
    ]
}

queries = [
    "Can I cancel a subscription service?",           # 関連する内容が存在する
    "What is the hiring process at this company?",     # 関連する内容がない
]
 
for query in queries:
    print(f"\nQuery: {query}")
    results = vector_store.similarity_search_with_score(query, k=1)
    for doc, score in results:
        print(f"  [dist={score:.4f}] {doc.page_content[:80]}...")

Query: Can I cancel a subscription service?
  [dist=0.7511] ## Subscription Services
Monthly subscriptions may be cancelled at any time. Re...
 
Query: What is the hiring process at this company?
  [dist=1.4648] ## Refund Policy
Refunds are available within 7 days of purchase for unopened i...

from langchain_openai import ChatOpenAI
 
llm = ChatOpenAI(model="gpt-5-mini")
 
def retrieve_or_abstain(query: str, max_distance: float = 1.0, k: int = 3):
    """距離のしきい値以下のチャンクのみを返します。1つも通過しない場合はNoneを返します。"""
    scored = vector_store.similarity_search_with_score(query, k=k)
    good = [doc for doc, dist in scored if dist <= max_distance]
    return good or None
 
def safe_answer(query: str) -> str:
    docs = retrieve_or_abstain(query)
    if docs is None:
        return "I don't have enough information to answer that question."
 
    context = "\n\n".join(d.page_content for d in docs)
    prompt = (
        "Answer the question using ONLY the context below.\n\n"
        f"Context:\n{context}\n\nQuestion: {query}"
    )
    return llm.invoke(prompt).content

# 関連する内容がある質問
print(safe_answer("Can I cancel a subscription service?"))
print("---")
# 関連する内容がない質問
print(safe_answer("What is the hiring process at this company?"))

Yes. Monthly subscriptions may be cancelled at any time, and refunds are prorated
based on the remaining days in the billing cycle. Annual subscriptions may be refunded
in full within the first 14 days...
---
I don't have enough information to answer that question.

results = vector_store.similarity_search_with_score(
    "What are the refund conditions?",
    k=10,
    filter={"category": "customer"},
)
 
for i, (doc, dist) in enumerate(results, 1):
    source = doc.metadata["source"].split("/")[-1]
    print(f"{i:>2}. [dist={dist:.4f}] [{source}] {doc.page_content[:80]}...")

 1. [dist=0.7798] [refund_policy.md] ## Standard Returns
**Effective Date**: January 1, 2026
All physical products ma...
 
 2. [dist=0.8203] [refund_policy.md] ## Defective Items
Defective items may be returned at any time for a full refun...
 
 3. [dist=0.9353] [refund_policy.md] ## Subscription Services
Monthly subscriptions may be cancelled at any time. Re...
 
 4. [dist=1.4249] [shipping_info.md] ## Lost or Damaged Packages
If your package is lost or arrives damaged, contact...
 
 5. [dist=1.5516] [shipping_info.md] ## Domestic Shipping
Standard shipping (5-7 business days): Free on orders over...
...

results_mmr = vector_store.max_marginal_relevance_search(
    "What are the refund conditions?",
    k=3,
    fetch_k=10,
)
 
for i, doc in enumerate(results_mmr):
    print(f"{i+1}. {doc.page_content[:80]}...")

1. ## Standard Returns
All physical products may be returned within 30 days of pur...
 
2. ## Defective Items
Defective items may be returned at any time for a full refun...
 
3. ## Digital Products
Digital products (software licenses, e-books, online course...