Skip to content

.Net: [MEVD] Implement store key generation where supported #11485

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
8 tasks
roji opened this issue Apr 10, 2025 · 0 comments
Open
8 tasks

.Net: [MEVD] Implement store key generation where supported #11485

roji opened this issue Apr 10, 2025 · 0 comments
Labels
msft.ext.vectordata Related to Microsoft.Extensions.VectorData .NET Issue or Pull requests regarding .NET code

Comments

@roji
Copy link
Member

roji commented Apr 10, 2025

Our upsert APIs allow returning keys/IDs which are generated in the vector store, but we don't actually implement that for connectors yet. We're also lacking a way for users to indicate that they want

  • Add IsAutoGenerated flag on VectorStoreRecordKeyProperty.
    • For the attribute, we could add it, or there's the .NET [DatabaseGenerated(DatabaseGeneratedOption.Identity)] (EF recognizes this, so do some other layers). Though the naming (identity) isn't ideal (it's SQL Server-specific).
  • Open design question: when a key property is configured as auto-generated, is it still possible for users to provide a value?
    • This opens up the question of how we determine whether the user has set the value or not. For this case, EF checks if the key property has the default value, and assumes it's "unset" if so; this works relatively well, as relational databases typically start from 1 for numbers, empty GUIDs (all-zeros) are also a useful sentinel value, etc.
    • But we can always start by not supporting this and add it later (i.e. if a key property is configured as auto-generated, it cannot be set).

Providers (see table here, we don't have to do them all at once):

  • MongoDB
  • CosmosNoSQL
  • CosmosMongoDB
  • PostgreSQL
  • Sqlite
  • SQL Server
  • Weaviate
  • Milvus (no IVectorStore connector yet)

See previous conversation in #10692.

@roji roji added .NET Issue or Pull requests regarding .NET code msft.ext.vectordata Related to Microsoft.Extensions.VectorData labels Apr 10, 2025
@github-actions github-actions bot changed the title [MEVD] Implement store key generation where supported .Net: [MEVD] Implement store key generation where supported Apr 10, 2025
@markwallace-microsoft markwallace-microsoft moved this to Backlog: Planned in Semantic Kernel Apr 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
msft.ext.vectordata Related to Microsoft.Extensions.VectorData .NET Issue or Pull requests regarding .NET code
Projects
Status: Backlog: Planned
Development

No branches or pull requests

2 participants