Table of Contents

Create a skill to answer questions about documents

Neos now relies on Microsoft.Extensions.VectorData for document vectorization and vector search.

What changed

  • IDocumentVectorizer is kept for retro-compatibility with the historical Kernel Memory behavior.
  • For new vectorization code, use IVectorDataDocumentVectorizer.
  • In IDocumentLookup, LookupAsync is the retro-compatible lookup method.
  • For new vector search code, use SearchAsync.

Supported backends

  • PostgreSQL with pgVector extension
  • Azure AI Search

For all new developments, inject GroupeIsa.Neos.Application.AI.Memories.IVectorDataDocumentVectorizer.

Use VectorizeDocumentAsync, VectorizeTextAsync, or VectorizeWebPageAsync depending on your source.

Warning

Starting with Neos 3.2, VectorizeDocumentAsync supports only PDF, Markdown, and plain text documents.
In previous versions, Office documents (Word, Excel, and PowerPoint) were also supported.

Example (document stream):

public Task ExecuteAsync(int documentId)
{
    Domain.Entities.Document document = _documentRepository.Get(documentId);
    if (document.Content == null)
    {
        throw new BusinessException("Document content is missing");
    }

    DocumentTagCollection tags = [];
    if (document.Category != null)
    {
        tags.Add("CATEGORY", document.Category);
    }

    return _vectorizer.VectorizeDocumentAsync(
        documentStream: new MemoryStream(document.Content.Content),
        documentId: document.DocumentIdentifier ?? documentId.ToString(),
        documentName: document.Name,
        vectorName: "documents",
        tags: tags,
        cancellationToken: default);
}

Main arguments:

  • modelId: Optional when using overloads with explicit embedding model; omit to use the default model configured in AI:MemoryDb:EmbeddingsModel.
  • documentStream or documentContent or url: Source to vectorize.
  • documentId: Unique identifier for the document.
  • vectorName: Name of the vector collection.
  • tags: Metadata used later for filtering.

Inject GroupeIsa.Neos.Application.AI.IDocumentLookup and use SearchAsync for new vector search scenarios.

[KernelFunction, Description("Search in documents")]
public async Task<string?> SearchDocumentsAsync(
    [Description("The question to ask about documents")] string question,
    [Description("The category of the document")] string? category = null)
{
    DocumentSearchOptions options = new()
    {
        ScoreThreshold  = 0.82,
    };

    if (!string.IsNullOrWhiteSpace(category))
    {
        options.Filter = DocumentFilters.ByTag("CATEGORY", category);
    }

    StringBuilder stringBuilder = new();
    foreach (var result in await _documentLookup.SearchAsync(
        embeddingModelId: "text-embedding-3-small",
        question: question,
        vectorName: "documents",
        options: options,
        cancellationToken: default))
    {
        stringBuilder.AppendLine($"# Extracts of document {result.Source}");
        foreach (var part in result.Parts.OrderBy(r => r.RelevantScore).Take(3))
        {
            stringBuilder.AppendLine(part.Text);
        }

        stringBuilder.AppendLine($"The source of the extract is {result.Source}.");
        stringBuilder.AppendLine();
    }

    return stringBuilder.ToString();
}

Main arguments:

  • embeddingModelId: Embedding model identifier used for the search.
  • question: User question.
  • vectorName: Vector collection name.
  • options: Search options, including filters and relevance threshold.

Retro-compatibility APIs

The following APIs are still available for compatibility with existing code based on Kernel Memory semantics:

  • GroupeIsa.Neos.Application.AI.Memories.IDocumentVectorizer
  • GroupeIsa.Neos.Application.AI.IDocumentLookup.LookupAsync
Warning

Azure AI Search is not supported by the Kernel Memory retro-compatibility path. Starting with Neos 3.2, when using Azure AI Search, you must use the Vector Data dedicated APIs: IVectorDataDocumentVectorizer for vectorization and IDocumentLookup.SearchAsync for search.

Use them only when maintaining legacy features. For new features, prefer IVectorDataDocumentVectorizer and SearchAsync.

Configuration

In your YAML configuration:

For PostgreSQL with pgVector extension:

AI:
  MemoryDb:
    ConnectorType: Postgres

For Azure AI Search:

AI:
  MemoryDb:
    ConnectorType: AzureAISearch

Other settings should be configured through dotnet user-secrets or environment variables.

For PostgreSQL with pgVector extension:

dotnet user-secrets set "AI:MemoryDb:ConnectionString" "Your connection string" --id "Your project id"

For Azure AI Search:

dotnet user-secrets set "AI:MemoryDb:ApiKey" "Your Azure AI Search administrator API key" --id "Your project id"
dotnet user-secrets set "AI:MemoryDb:Endpoint" "https://[your_deploy_name].search.windows.net" --id "Your project id"