Independent product-design exploration
Designing governable knowledge for enterprise AI
An enterprise AI knowledge system should make provenance, selection rules, operational state, and quality recovery understandable without forcing every user through code-level configuration.
- Information provenance
- Reusable scope rules
- Evaluation recovery
Context
The project requirements described a knowledge-management capability within a broader enterprise AI platform. Different teams would need to connect sources, organize governed content, inspect how documents were processed, evaluate retrieval quality, and control what knowledge an agent could use.
The design challenge was not only to expose technical configuration. It was to make provenance, permissions, freshness, retrieval behavior, and failure recovery legible to people with different levels of technical depth.
My responsibility
I was the sole designer for this discovery-and-definition project. Starting from the supplied project requirements, I created the information architecture, detailed flows and states, wireframes, visual system, local component set, and prototype. The working file contains the three required low-fidelity flows and a detailed high-fidelity source-connection sequence.
This was an independent product-design exploration, not employment or delivered product work.
Supplied assumptions
The project requirements assumed a large document corpus and many connected sources, mixed technical and operational roles, hierarchical permissions, multi-region and tenant constraints, PII controls, document versioning, asynchronous synchronization, and configurable retrieval behavior.
Those conditions define the problem space. They do not establish that the product existed at this scale or that any proposed behavior was implemented.
Where to focus within the knowledge platform
Knowledge-platform scope
Source quality determines what an agent can reference.
Reusable rules make knowledge scope manageable as sources change.
Failures need a path to evidence, correction and a new test.
EAI-01 · Representative graphic
- Source connection
- Project requirements A source-connection flow.
- My design decisions Low-fidelity exploration and a high-fidelity sequence detailing configuration, states and recovery.
- Collection creation
- Project requirements Rules, permissions and agent attachment.
- My design decisions A low-fidelity compound inclusion/exclusion model with matched-content feedback.
- Evaluation
- Project requirements A failed evaluation leading to a document or policy improvement.
- My design decisions A low-fidelity evidence drill-down with corrective actions and rerun continuity.
Provenance
Decision 01: Start with information provenance
The project requirements included a source-connection flow and allowed source setup as a high-fidelity choice. I selected it for the high-fidelity focus because an agent’s answer quality depends on the material it can reference. Source setup was therefore treated as product behavior rather than background administration.
The proposed sequence moved through connector choice, authentication, source identity, file or folder scope, parsing and retrieval configuration, and indexing. A guided sequence reduced the number of simultaneous decisions while keeping the technical choices that affect downstream knowledge quality available.
The source overview then made ownership, synchronization state, and last-update information visible. The intended effect was to help teams understand where knowledge came from, whether it was current, and what processing state they should expect before relying on it.
Complete source-connection flow
1. Connect
Allow read-only access to the documents you choose.
2. Select assets
Selected folders remain connected as their contents change.
3. Configure
EAI-02 · Representative graphic
- Connect and define scope. Select a connector, authenticate, identify the source and choose the files or folders it can reference.
- Retain consequential configuration. Keep parsing and retrieval choices available before indexing, with deeper controls reachable through advanced settings.
- Make processing and recovery visible. Distinguish synchronization from errors and provide the proposed retry path.
- Inspect operational state. Show source ownership, synchronization state and last-update information before relying on the knowledge.
Collection rules
Decision 02: Use reusable rules instead of persistent item-by-item selection
Individual document selection would become difficult to maintain under the project’s assumed scale and ongoing synchronization. I proposed compound inclusion and exclusion rules based on file properties or metadata, allowing a collection’s scope to remain inspectable as sources changed.
The alternative was direct selection of individual documents. That could be precise for a small, static set, but it would create recurring manual work and make coverage harder to reason about as the corpus evolved.
The intended effect of reusable rules was to move repeated governance from item-level maintenance into an explicit, reviewable definition. This effect was not tested with users.
Inspect rule-based collection scope
Collection scope
New matches enter the collection. Files that stop matching leave it. Selection does not grant additional source permissions.
EAI-03 · Representative graphic
- Individual selection
- Precise for a small, static set, but requires repeated item-level maintenance as sources change.
- Reusable rules
- Compound inclusion and exclusion conditions based on file properties or metadata define an inspectable scope.
- Matched content
- The proposed feedback shows what the rule matches. Whether users can understand and configure the rules remains untested.
Control depth
Decision 03: Guide common configuration without hiding expert control
The project requirements included engineers, product managers, and operations roles. I assumed they would not all need the same depth at the same moment.
The response exposed common decisions and safe defaults in the primary flow, while placing controls such as retrieval weighting and relevance thresholds in advanced settings. This proposed a progressive path: routine users could complete a bounded task directly, while specialists could still inspect and tune consequential system behavior.
This role model was an exercise assumption that would need research. The response did not establish that the defaults, labels, or control depth were usable.
Evaluation
Decision 04: Turn evaluation into a debugging and recovery loop
The project requirements called for a failing evaluation case to lead to a document or policy improvement. I elaborated that skeleton into a debugging path: compare a query with an expected answer, trace the result to source chunks and relevance evidence, and connect failure to corrective actions such as excluding a problematic chunk, revising the expected answer, or reopening the collection.
Tests could then be rerun after a knowledge change. The intended effect was to preserve evaluation as reusable organizational knowledge and make quality failures diagnosable rather than terminal.
Accuracy, hallucination, latency, trends, and sharing were proposed evaluation concepts. Their usefulness, definitions, and thresholds remained assumptions requiring validation.
Evaluation results, evidence and correction
1. Review results
Needs reviewOpen a result to compare the expected answer with the agent response.
2. Inspect evidence
3. Choose a correction
“…replace after twelve months…”
Choose the correction supported by the evidence, then run the test again.
EAI-04 · Representative graphic
- Identify the failure. Compare the query and expected answer with the evaluation result.
- Inspect evidence. Trace the result to source chunks and relevance evidence.
- Choose a proposed correction. Exclude a problematic chunk, revise the expected answer or reopen the collection.
- Rerun the evaluation. Retain continuity after a knowledge change. No successful recovery or improvement is established.
Reflection
What I would validate next
- Whether each target role recognizes the source, collection, and evaluation concepts and can predict their consequences.
- Whether the guided source flow exposes the right defaults and makes advanced configuration discoverable without making it mandatory.
- Whether collection rules are understandable before they run, previewable against real content, and recoverable when they match too much or too little.
- Whether ownership, synchronization, freshness, restricted access, empty results, processing, and failure states are distinguishable with keyboard and screen-reader use.
- Whether tracing a failed answer to source evidence leads people to a correct next action rather than encouraging unsafe content changes.
- Which evaluation concepts and trend views support actual operational decisions, and which add noise or false confidence.
These are proposed validation questions, not research findings.
What this work demonstrates
The exploration demonstrates how I reason about enterprise AI as an operational system: begin with provenance, convert repeated governance into inspectable rules, match control depth to the task, and make quality failures lead to diagnosis and recovery.
It demonstrates systems and interaction judgment only. It does not demonstrate cross-functional delivery, production performance, user validation, organizational adoption, or business impact.
The representative graphics use neutral illustrative content. They demonstrate proposed behavior, not a working product or measured results.