Decision: Adopt NDC‑sharded knowledge architecture with bilingual support, provider‑agnostic AI proxy, and governance safeguards

Context#

• Recent work introduced a reorganization of indices into NDC-aligned shards and expanded knowledge materials on bilingual (JP–EN) indexing standards and controlled vocabulary crosswalks aligned to NDC.
• Multilingual output support has been improved, with explicit intent to test.
• The AI proxy layer includes provider integrations for major vendors (e.g., Google and OpenAI) and shared system prompt handling.
• Knowledge resources were added covering risk-based AI governance, trustworthy AI characteristics (e.g., per widely accepted frameworks), AI incident response runbooks, and compliance topics such as APPI cross-border data transfer consent and adequacy/white-listing.
• A self-recognition evaluation environment expanded, with materials on inner speech benefits and ablation testing strategies to isolate its contribution.

Decision summary#

Proceed with Phase 1 adoption of NDC-sharded knowledge indexing and JP–EN bilingual retrieval, standardize on the provider-agnostic AI proxy for Google/OpenAI, and integrate governance, incident runbooks, and APPI-aligned practices; prioritize self-recognition evaluation with targeted ablation testing for inner speech.

Alignment points#

• Knowledge architecture: Aligns retrieval to established NDC taxonomies and bilingual crosswalks.
• Product capability: Multilingual output support is explicitly being improved and tested.
• Platform strategy: Provider-neutral AI proxy consolidates integrations while supporting major vendors.
• Risk & compliance: Incorporates risk-based AI governance, trustworthy AI principles, incident response, and APPI cross-border guidance.
• R&D rigor: Inner speech evaluation through ablation aligns with evidence-based improvement.

Disagreements (with reasons)#

• Breadth vs. depth: Some prefer deeper monolingual optimization before broadening to bilingual; others prioritize bilingual now to match user needs.
• Abstraction vs. specialization: A provider-agnostic proxy simplifies operations, but may delay adoption of vendor-specific features.
• Sharding complexity: NDC sharding can increase operational complexity; proponents argue taxonomy-aligned retrieval offsets this with better precision.

Recommended decision range#

• Scope: Limit Phase 1 to knowledge retrieval and evaluation areas; restrict languages to JP–EN and providers to Google/OpenAI.
• Governance: Adopt risk-based AI governance materials, trustworthy AI characteristics, incident runbooks, and APPI cross-border consent guidance.
• R&D: Execute inner speech ablation tests within the self-recognition evaluation environment.

Risks and mitigations#

1) Retrieval regressions from NDC sharding

• Mitigation: Establish baselines and run ablation-style comparisons to quantify retrieval quality changes.

2) Multilingual inconsistency (JP–EN)

• Mitigation: Define a targeted test suite and spot-checks for bilingual indexing and output; iterate on identified failure modes.

3) Compliance gaps in cross-border handling (APPI)

• Mitigation: Use informed consent requirements and adequacy/white-list guidance; document and review disclosures before rollout.

Assumptions (top 3)#

• NDC taxonomy coverage and the JP–EN crosswalks are sufficient for initial retrieval improvements.
• Provider APIs for Google and OpenAI remain stable under the AI proxy abstraction.
• The organization will adopt risk-based AI governance, trustworthy AI practices, and incident response runbooks.

KPI impact assumptions#

• Retrieval quality: Directional improvement in taxonomy-aligned relevance and consistency.
• Multilingual quality: Higher JP–EN answer accuracy and reduced inconsistency.
• Platform reliability: More predictable request routing and error handling across providers.
• Risk posture: Reduced compliance exposure and faster incident detection-to-mitigation.

Next steps checklist (max 5)#

• Validate retrieval quality post–NDC sharding using baseline and ablation comparisons.
• Execute JP–EN multilingual output tests and remediate gaps.
• Run inner speech ablation experiments in the self-recognition evaluation environment and document findings.
• Exercise AI proxy routing across Google/OpenAI, verifying fallbacks and logging.
• Review APPI cross-border consent notices and incident runbooks with stakeholders; finalize adoption plan.