diff --git a/docs/concepts/mcp.md b/docs/concepts/mcp.md index 6a82ba73..9a6970e5 100644 --- a/docs/concepts/mcp.md +++ b/docs/concepts/mcp.md @@ -15,7 +15,7 @@ The RedisVL MCP server sits between an MCP client and Redis: 1. It connects to an existing Redis Search index. 2. It inspects that index at startup and reconstructs its schema. -3. It instantiates the configured vectorizer for query embedding and optional upsert embedding. +3. It initializes vector capabilities only when the configured search or upsert behavior needs them. 4. It exposes stable MCP tools for search, and optionally upsert. This keeps the Redis index as the source of truth for search behavior while giving MCP clients a predictable interface. @@ -27,7 +27,7 @@ RedisVL MCP works with a focused model: - One server process binds to exactly one existing Redis index. - The server supports stdio (default), Streamable HTTP, and SSE transports. - Search behavior is owned by configuration, not by MCP callers. -- The vectorizer is configured explicitly. +- Vector search and server-side embedding are optional capabilities configured explicitly. - Upsert is optional and can be disabled with read-only mode. ## Config-Owned Search Behavior @@ -85,7 +85,7 @@ Use read-only mode when Redis is serving approved content to assistants and anot RedisVL MCP exposes two tools: - `search-records` searches the configured index using the server-owned search mode -- `upsert-records` validates and upserts records, embedding them when needed +- `upsert-records` validates and upserts records, embedding them only when that capability is configured These tools follow a stable contract: diff --git a/docs/user_guide/how_to_guides/mcp.md b/docs/user_guide/how_to_guides/mcp.md index b8dcecd6..5a1755aa 100644 --- a/docs/user_guide/how_to_guides/mcp.md +++ b/docs/user_guide/how_to_guides/mcp.md @@ -161,12 +161,43 @@ indexes: - `redis_name` must point to an index that already exists in Redis - `search.type` fixes retrieval behavior for every MCP caller -- `runtime.text_field_name` tells full-text and hybrid search which field to search -- `runtime.vector_field_name` tells the server which vector field to use -- `runtime.default_embed_text_field` tells upsert which text field to embed when a record needs embedding +- `runtime.text_field_name` is required for `fulltext` and `hybrid` search +- `runtime.vector_field_name` is required for `vector` and `hybrid` search, and optional for plain full-text deployments +- `runtime.default_embed_text_field` is only required when the server should generate embeddings during upsert +- `vectorizer` is required for query embedding and server-side embedding, but optional for fulltext-only configs - `runtime.max_result_window` caps deep paging by limiting the maximum `offset + limit` - `schema_overrides` is only for patching incomplete field attrs discovered from Redis +### Fulltext-Only Config + +For a non-vector deployment, omit vector-only settings entirely: + +```yaml +server: + redis_url: ${REDIS_URL} + +indexes: + knowledge: + redis_name: knowledge + + search: + type: fulltext + params: + text_scorer: BM25STD + stopwords: english + + runtime: + text_field_name: content + default_limit: 10 + max_limit: 25 + max_result_window: 1000 + max_upsert_records: 64 + skip_embedding_if_present: true + startup_timeout_seconds: 30 + request_timeout_seconds: 60 + max_concurrency: 16 +``` + ## Tool Contracts RedisVL MCP exposes a small, implementation-owned contract. @@ -269,8 +300,9 @@ Example response payload: Notes: - this tool is not registered in read-only mode -- records that need embedding must contain `runtime.default_embed_text_field` -- when `skip_embedding_if_present` is `true`, records that already contain the vector field can skip re-embedding +- when server-side embedding is configured, records that need embedding must contain `runtime.default_embed_text_field` +- when `skip_embedding_if_present` is `true`, records that already contain the configured vector field can skip re-embedding +- when a vector field is configured but server-side embedding is disabled, callers must supply vectors explicitly ## Search Examples @@ -425,6 +457,24 @@ If a record does not include the configured vector field, RedisVL MCP embeds `ru Set `skip_embedding_if_present` to `false` when you want the server to regenerate embeddings during upsert. In most cases, the caller should omit the vector field and let the server manage embeddings from `runtime.default_embed_text_field`. +### Plain Writes Without Embedding + +For fulltext-only indexes, `upsert-records` can write records without any vectorizer or vector field configuration: + +```json +{ + "records": [ + { + "content": "Updated FAQ entry", + "category": "support", + "rating": 5 + } + ] +} +``` + +If you configure a vector field but omit server-side embedding support, the caller must send vectors in each record instead of relying on the server to generate them. + ## Troubleshooting ### Missing MCP Dependencies @@ -435,7 +485,7 @@ If `rvl mcp` reports missing optional dependencies, install the MCP extra: pip install redisvl[mcp] ``` -If the configured vectorizer needs a provider SDK, install that provider extra too. +If the configured vectorizer needs a provider SDK, install that provider extra too. Fulltext-only configs can omit the vectorizer entirely. ### Configured Redis Index Does Not Exist diff --git a/redisvl/mcp/config.py b/redisvl/mcp/config.py index aa937caf..d1ae8cbc 100644 --- a/redisvl/mcp/config.py +++ b/redisvl/mcp/config.py @@ -34,9 +34,9 @@ def reserved_score_metadata_field_names() -> frozenset[str]: class MCPRuntimeConfig(BaseModel): """Runtime limits and validated field mappings for MCP requests.""" - text_field_name: str = Field(..., min_length=1) - vector_field_name: str = Field(..., min_length=1) - default_embed_text_field: str = Field(..., min_length=1) + text_field_name: str | None = Field(default=None, min_length=1) + vector_field_name: str | None = Field(default=None, min_length=1) + default_embed_text_field: str | None = Field(default=None, min_length=1) default_limit: int = 10 max_limit: int = 100 max_result_window: int = 1000 @@ -209,11 +209,76 @@ class MCPIndexBindingConfig(BaseModel): """The sole configured v1 index binding.""" redis_name: str = Field(..., min_length=1) - vectorizer: MCPVectorizerConfig + vectorizer: MCPVectorizerConfig | None = None search: MCPIndexSearchConfig runtime: MCPRuntimeConfig schema_overrides: MCPSchemaOverrides = Field(default_factory=MCPSchemaOverrides) + @property + def uses_text_search(self) -> bool: + """Return whether search queries depend on a configured text field.""" + return self.search.type in {"fulltext", "hybrid"} + + @property + def uses_query_embedding(self) -> bool: + """Return whether search queries require embedding the user's query.""" + return self.search.type in {"vector", "hybrid"} + + @property + def supports_vector_backed_upsert(self) -> bool: + """Return whether upsert should manage a configured vector field.""" + return self.runtime.vector_field_name is not None + + @property + def supports_server_side_embedding(self) -> bool: + """Return whether upsert can generate vectors from text fields.""" + return ( + self.runtime.vector_field_name is not None + and self.runtime.default_embed_text_field is not None + and self.vectorizer is not None + ) + + @property + def requires_startup_vectorizer(self) -> bool: + """Return whether startup must initialize a configured vectorizer.""" + return self.uses_query_embedding or self.supports_server_side_embedding + + @model_validator(mode="after") + def _validate_capability_requirements(self) -> "MCPIndexBindingConfig": + """Require only the config fields needed by enabled capabilities.""" + if self.uses_text_search and self.runtime.text_field_name is None: + raise ValueError( + "runtime.text_field_name is required for " + f"search.type '{self.search.type}'" + ) + + if self.uses_query_embedding and self.runtime.vector_field_name is None: + raise ValueError( + "runtime.vector_field_name is required for " + f"search.type '{self.search.type}'" + ) + + if self.uses_query_embedding and self.vectorizer is None: + raise ValueError( + f"vectorizer is required for search.type '{self.search.type}'" + ) + + if ( + self.runtime.default_embed_text_field is not None + and self.runtime.vector_field_name is None + ): + raise ValueError( + "runtime.default_embed_text_field requires runtime.vector_field_name" + ) + + if ( + self.runtime.default_embed_text_field is not None + and self.vectorizer is None + ): + raise ValueError("runtime.default_embed_text_field requires vectorizer") + + return self + class MCPConfig(BaseModel): """Validated MCP server configuration loaded from YAML.""" @@ -250,7 +315,7 @@ def runtime(self) -> MCPRuntimeConfig: return self.binding.runtime @property - def vectorizer(self) -> MCPVectorizerConfig: + def vectorizer(self) -> MCPVectorizerConfig | None: """Expose the sole binding's vectorizer config for phase 1.""" return self.binding.vectorizer @@ -259,6 +324,31 @@ def search(self) -> MCPIndexSearchConfig: """Expose the sole binding's configured search behavior.""" return self.binding.search + @property + def uses_text_search(self) -> bool: + """Return whether configured search uses a text field.""" + return self.binding.uses_text_search + + @property + def uses_query_embedding(self) -> bool: + """Return whether configured search embeds user queries.""" + return self.binding.uses_query_embedding + + @property + def supports_vector_backed_upsert(self) -> bool: + """Return whether configured upserts manage a vector field.""" + return self.binding.supports_vector_backed_upsert + + @property + def supports_server_side_embedding(self) -> bool: + """Return whether configured upserts can generate embeddings.""" + return self.binding.supports_server_side_embedding + + @property + def requires_startup_vectorizer(self) -> bool: + """Return whether startup must initialize a vectorizer.""" + return self.binding.requires_startup_vectorizer + @property def redis_name(self) -> str: """Return the existing Redis index name that must be inspected at startup.""" @@ -343,26 +433,33 @@ def validate_runtime_mapping(self, schema: IndexSchema) -> None: """Ensure runtime mappings point at explicit fields in the effective schema.""" field_names = set(schema.field_names) - if self.runtime.text_field_name not in field_names: + if self.uses_text_search and self.runtime.text_field_name not in field_names: raise ValueError( f"runtime.text_field_name '{self.runtime.text_field_name}' not found in schema" ) - if self.runtime.default_embed_text_field not in field_names: + if ( + self.supports_server_side_embedding + and self.runtime.default_embed_text_field not in field_names + ): raise ValueError( "runtime.default_embed_text_field " f"'{self.runtime.default_embed_text_field}' not found in schema" ) - vector_field = schema.fields.get(self.runtime.vector_field_name) - if vector_field is None: - raise ValueError( - f"runtime.vector_field_name '{self.runtime.vector_field_name}' not found in schema" - ) - if vector_field.type != "vector": - raise ValueError( - f"runtime.vector_field_name '{self.runtime.vector_field_name}' must reference a vector field" - ) + if self.uses_query_embedding or self.supports_vector_backed_upsert: + vector_field_name = self.runtime.vector_field_name + if vector_field_name is None: + raise ValueError("runtime.vector_field_name is not configured") + vector_field = schema.fields.get(vector_field_name) + if vector_field is None: + raise ValueError( + f"runtime.vector_field_name '{vector_field_name}' not found in schema" + ) + if vector_field.type != "vector": + raise ValueError( + f"runtime.vector_field_name '{vector_field_name}' must reference a vector field" + ) def to_index_schema(self, inspected_schema: dict[str, Any]) -> IndexSchema: """Apply overrides to an inspected schema and validate the effective result.""" @@ -373,6 +470,8 @@ def to_index_schema(self, inspected_schema: dict[str, Any]) -> IndexSchema: def get_vector_field(self, schema: IndexSchema) -> BaseField: """Return the effective vector field from a validated schema.""" + if self.runtime.vector_field_name is None: + raise ValueError("runtime.vector_field_name is not configured") return schema.fields[self.runtime.vector_field_name] def get_vector_field_dims(self, schema: IndexSchema) -> int | None: diff --git a/redisvl/mcp/server.py b/redisvl/mcp/server.py index 4a5f5069..2e5adb2b 100644 --- a/redisvl/mcp/server.py +++ b/redisvl/mcp/server.py @@ -106,8 +106,10 @@ async def get_index(self) -> AsyncSearchIndex: async def get_vectorizer(self) -> Any: """Return the initialized vectorizer or fail if startup has not run.""" - if self._vectorizer is None: + if self.config is None: raise RuntimeError("MCP server has not been started") + if self._vectorizer is None: + raise RuntimeError("MCP server vectorizer is not configured") return self._vectorizer async def run_guarded(self, operation_name: str, awaitable: Awaitable[Any]) -> Any: @@ -147,6 +149,8 @@ def _build_vectorizer(self) -> Any: """Instantiate the configured vectorizer class from validated config.""" if self.config is None: raise RuntimeError("MCP server config not loaded") + if self.config.vectorizer is None: + raise RuntimeError("MCP server vectorizer is not configured") vectorizer_class = resolve_vectorizer_class(self.config.vectorizer.class_name) return vectorizer_class(**self.config.vectorizer.to_init_kwargs()) @@ -298,7 +302,8 @@ async def _initialize_runtime_resources(self) -> Any: schema=effective_schema, supports_native_hybrid_search=await self.supports_native_hybrid_search(), ) - await self._initialize_vectorizer(effective_schema, timeout) + if self.config.requires_startup_vectorizer: + await self._initialize_vectorizer(effective_schema, timeout) self._register_tools(effective_schema) return client except Exception: diff --git a/redisvl/mcp/tools/search.py b/redisvl/mcp/tools/search.py index 3d96e669..00ef02d6 100644 --- a/redisvl/mcp/tools/search.py +++ b/redisvl/mcp/tools/search.py @@ -125,13 +125,17 @@ def _validate_request( ) schema_fields = set(index.schema.field_names) - vector_field_name = runtime.vector_field_name + vector_field_names = { + field_name + for field_name, field in index.schema.fields.items() + if field.type == "vector" + } if return_fields is None: fields = [ field_name for field_name in index.schema.field_names - if field_name != vector_field_name + if field_name not in vector_field_names ] else: if not isinstance(return_fields, list): @@ -154,9 +158,9 @@ def _validate_request( code=MCPErrorCode.INVALID_REQUEST, retryable=False, ) - if field_name == vector_field_name: + if field_name in vector_field_names: raise RedisVLMCPError( - f"Vector field '{vector_field_name}' cannot be returned", + f"Vector field '{field_name}' cannot be returned", code=MCPErrorCode.INVALID_REQUEST, retryable=False, ) @@ -236,6 +240,9 @@ def _build_native_hybrid_kwargs( search_params: dict[str, Any], ) -> dict[str, Any]: """Build native `HybridQuery` kwargs from MCP config-owned hybrid params.""" + if runtime.text_field_name is None or runtime.vector_field_name is None: + raise RuntimeError("Hybrid search requires configured text and vector fields") + params = dict(search_params) combination_method = params.setdefault( "combination_method", @@ -276,6 +283,9 @@ def _build_fallback_hybrid_kwargs( search_params: dict[str, Any], ) -> dict[str, Any]: """Build aggregate fallback kwargs while preserving MCP fusion semantics.""" + if runtime.text_field_name is None or runtime.vector_field_name is None: + raise RuntimeError("Hybrid search requires configured text and vector fields") + params = dict(search_params) linear_text_weight = params.pop( "linear_text_weight", @@ -319,6 +329,8 @@ async def _build_query( filter_expression = parse_filter(filter_value, index.schema) if search_type == "vector": + if runtime.vector_field_name is None: + raise RuntimeError("Vector search requires a configured vector field") vectorizer = await server.get_vectorizer() embedding = await _embed_query(vectorizer, query) vector_kwargs = { @@ -344,6 +356,8 @@ async def _build_query( ) if search_type == "fulltext": + if runtime.text_field_name is None: + raise RuntimeError("Full-text search requires a configured text field") return ( TextQuery( text=query, diff --git a/redisvl/mcp/tools/upsert.py b/redisvl/mcp/tools/upsert.py index aedb8f34..293a1119 100644 --- a/redisvl/mcp/tools/upsert.py +++ b/redisvl/mcp/tools/upsert.py @@ -73,10 +73,12 @@ def _validate_request( def _record_needs_embedding( record: dict[str, Any], *, - vector_field_name: str, + vector_field_name: str | None, skip_embedding_if_present: bool, ) -> bool: """Determine whether a record requires server-side embedding.""" + if vector_field_name is None: + return False return ( not skip_embedding_if_present or vector_field_name not in record @@ -114,6 +116,31 @@ def _validate_embed_sources( return contents +def _validate_supplied_vectors( + records: list[dict[str, Any]], + *, + vector_field_name: str, + skip_embedding_if_present: bool, +) -> None: + """Require explicit vectors when embedding support is not configured.""" + missing_vector_count = 0 + for record in records: + if _record_needs_embedding( + record, + vector_field_name=vector_field_name, + skip_embedding_if_present=skip_embedding_if_present, + ): + missing_vector_count += 1 + + if missing_vector_count: + raise RedisVLMCPError( + "records requiring vector-backed upsert must include a non-empty " + f"'{vector_field_name}' field when server-side embedding is disabled", + code=MCPErrorCode.INVALID_REQUEST, + retryable=False, + ) + + async def _embed_one(vectorizer: Any, content: str) -> list[float]: """Embed one record, falling back from async to sync implementations.""" aembed = getattr(vectorizer, "aembed", None) @@ -165,12 +192,14 @@ def _vector_dtype(server: Any, index: Any) -> str: def _validation_schema_for_record( index: Any, *, - vector_field_name: str, + vector_field_name: str | None, record: dict[str, Any], ) -> Any: """Use a JSON-shaped schema when validating list vectors for HASH storage.""" - if index.schema.index.storage_type == StorageType.HASH and isinstance( - record.get(vector_field_name), list + if ( + vector_field_name is not None + and index.schema.index.storage_type == StorageType.HASH + and isinstance(record.get(vector_field_name), list) ): schema = index.schema.model_copy(deep=True) schema.index.storage_type = StorageType.JSON @@ -179,7 +208,7 @@ def _validation_schema_for_record( def _validate_record( - record: dict[str, Any], *, index: Any, vector_field_name: str + record: dict[str, Any], *, index: Any, vector_field_name: str | None ) -> None: """Validate one record against the schema, allowing HASH list vectors.""" validate_object( @@ -203,6 +232,9 @@ def _prepare_record_for_storage( vector_field_name = server.config.runtime.vector_field_name _validate_record(prepared, index=index, vector_field_name=vector_field_name) + if vector_field_name is None: + return prepared + vector_value = prepared.get(vector_field_name) if index.schema.index.storage_type == StorageType.HASH: @@ -240,27 +272,50 @@ async def upsert_records( index=index, vector_field_name=runtime.vector_field_name, ) - embed_contents = _validate_embed_sources( - prepared_records, - embed_text_field=runtime.default_embed_text_field, - vector_field_name=runtime.vector_field_name, - skip_embedding_if_present=effective_skip_embedding, - ) + if server.config.supports_server_side_embedding: + if ( + runtime.default_embed_text_field is None + or runtime.vector_field_name is None + ): + raise RuntimeError( + "Server-side embedding requires configured text and vector fields" + ) + embed_contents = _validate_embed_sources( + prepared_records, + embed_text_field=runtime.default_embed_text_field, + vector_field_name=runtime.vector_field_name, + skip_embedding_if_present=effective_skip_embedding, + ) - if embed_contents: - vectorizer = await server.get_vectorizer() - embeddings = await _embed_many(vectorizer, embed_contents) - # Tracks position in the compact embeddings list, which only contains - # vectors for records that still need server-side embedding. - embedding_index = 0 - for record in prepared_records: - if _record_needs_embedding( - record, - vector_field_name=runtime.vector_field_name, - skip_embedding_if_present=effective_skip_embedding, - ): - record[runtime.vector_field_name] = embeddings[embedding_index] - embedding_index += 1 + if embed_contents: + vectorizer = await server.get_vectorizer() + # TODO: Avoid re-embedding records that already include vectors. + # The current flow can regenerate embeddings for caller-supplied + # vectors, which is wasteful and can add external service cost. + embeddings = await _embed_many(vectorizer, embed_contents) + # Tracks position in the compact embeddings list, which only contains + # vectors for records that still need server-side embedding. + embedding_index = 0 + for record in prepared_records: + if _record_needs_embedding( + record, + vector_field_name=runtime.vector_field_name, + skip_embedding_if_present=effective_skip_embedding, + ): + record[runtime.vector_field_name] = embeddings[embedding_index] + embedding_index += 1 + elif runtime.vector_field_name is not None: + if not effective_skip_embedding: + raise RedisVLMCPError( + "skip_embedding_if_present=false requires server-side embedding support", + code=MCPErrorCode.INVALID_REQUEST, + retryable=False, + ) + _validate_supplied_vectors( + prepared_records, + vector_field_name=runtime.vector_field_name, + skip_embedding_if_present=effective_skip_embedding, + ) loadable_records = [ _prepare_record_for_storage(record, server=server, index=index) diff --git a/tests/integration/test_mcp/test_search_tool.py b/tests/integration/test_mcp/test_search_tool.py index a5eaf8f3..a59f11c9 100644 --- a/tests/integration/test_mcp/test_search_tool.py +++ b/tests/integration/test_mcp/test_search_tool.py @@ -92,29 +92,80 @@ def preprocess(record: dict) -> dict: await index.delete(drop=True) +@pytest.fixture +async def fulltext_only_index(async_client, worker_id): + schema = IndexSchema.from_dict( + { + "index": { + "name": f"mcp-fulltext-search-{worker_id}", + "prefix": f"mcp-fulltext-search:{worker_id}", + "storage_type": "hash", + }, + "fields": [ + {"name": "content", "type": "text"}, + {"name": "category", "type": "tag"}, + {"name": "rating", "type": "numeric"}, + ], + } + ) + index = AsyncSearchIndex(schema=schema, redis_client=async_client) + await index.create(overwrite=True, drop=True) + await index.load( + [ + { + "id": f"doc:{worker_id}:1", + "content": "science article about planets", + "category": "science", + "rating": 5, + }, + { + "id": f"doc:{worker_id}:2", + "content": "medical science and health", + "category": "health", + "rating": 4, + }, + ] + ) + + yield index + + await index.delete(drop=True) + + @pytest.fixture def mcp_config_path(tmp_path: Path, redis_url: str): - def factory(redis_name: str, search: dict) -> str: + def factory( + redis_name: str, + search: dict, + *, + runtime_overrides: dict | None = None, + include_vectorizer: bool = True, + ) -> str: + runtime = { + "text_field_name": "content", + "vector_field_name": "embedding", + "default_embed_text_field": "content", + "default_limit": 2, + "max_limit": 5, + } + if runtime_overrides: + runtime.update(runtime_overrides) + + binding = { + "redis_name": redis_name, + "search": search, + "runtime": runtime, + } + if include_vectorizer: + binding["vectorizer"] = { + "class": "FakeVectorizer", + "model": "fake-model", + "dims": 3, + } + config = { "server": {"redis_url": redis_url}, - "indexes": { - "knowledge": { - "redis_name": redis_name, - "vectorizer": { - "class": "FakeVectorizer", - "model": "fake-model", - "dims": 3, - }, - "search": search, - "runtime": { - "text_field_name": "content", - "vector_field_name": "embedding", - "default_embed_text_field": "content", - "default_limit": 2, - "max_limit": 5, - }, - } - }, + "indexes": {"knowledge": binding}, } config_path = tmp_path / f"{redis_name}-{search['type']}.yaml" config_path.write_text(yaml.safe_dump(config), encoding="utf-8") @@ -130,10 +181,21 @@ async def started_server(monkeypatch, searchable_index, mcp_config_path): lambda class_name: FakeVectorizer, ) - async def factory(search: dict) -> RedisVLMCPServer: + async def factory( + search: dict, + *, + redis_name: str | None = None, + runtime_overrides: dict | None = None, + include_vectorizer: bool = True, + ) -> RedisVLMCPServer: server = RedisVLMCPServer( MCPSettings( - config=mcp_config_path(searchable_index.schema.index.name, search) + config=mcp_config_path( + redis_name or searchable_index.schema.index.name, + search, + runtime_overrides=runtime_overrides, + include_vectorizer=include_vectorizer, + ) ) ) await server.startup() @@ -141,8 +203,8 @@ async def factory(search: dict) -> RedisVLMCPServer: servers = [] - async def started(search: dict) -> RedisVLMCPServer: - server = await factory(search) + async def started(search: dict, **kwargs) -> RedisVLMCPServer: + server = await factory(search, **kwargs) servers.append(server) return server @@ -204,6 +266,32 @@ async def test_search_records_fulltext_success(started_server): assert "science" in response["results"][0]["record"]["content"] +@pytest.mark.asyncio +async def test_search_records_fulltext_success_without_vector_configuration( + started_server, fulltext_only_index +): + server = await started_server( + {"type": "fulltext", "params": {"stopwords": None}}, + redis_name=fulltext_only_index.schema.index.name, + runtime_overrides={ + "vector_field_name": None, + "default_embed_text_field": None, + }, + include_vectorizer=False, + ) + + response = await search_records( + server, + query="science", + return_fields=["content", "category"], + ) + + assert response["search_type"] == "fulltext" + assert response["results"] + assert response["results"][0]["score_type"] == "text_score" + assert "science" in response["results"][0]["record"]["content"] + + @pytest.mark.asyncio async def test_search_records_respects_raw_string_filter(started_server): server = await started_server({"type": "vector"}) diff --git a/tests/integration/test_mcp/test_server_startup.py b/tests/integration/test_mcp/test_server_startup.py index ad0de0b3..6b9c315a 100644 --- a/tests/integration/test_mcp/test_server_startup.py +++ b/tests/integration/test_mcp/test_server_startup.py @@ -84,6 +84,7 @@ def factory( schema_overrides: dict[str, Any] | None = None, runtime_overrides: dict[str, Any] | None = None, search: dict[str, Any] | None = None, + include_vectorizer: bool = True, ) -> str: runtime = { "text_field_name": "content", @@ -93,20 +94,21 @@ def factory( if runtime_overrides: runtime.update(runtime_overrides) + binding = { + "redis_name": redis_name, + "search": search or {"type": "vector"}, + "runtime": runtime, + } + if include_vectorizer: + binding["vectorizer"] = { + "class": "FakeVectorizer", + "model": "fake-model", + "dims": vector_dims, + } + config = { "server": {"redis_url": redis_url}, - "indexes": { - "knowledge": { - "redis_name": redis_name, - "vectorizer": { - "class": "FakeVectorizer", - "model": "fake-model", - "dims": vector_dims, - }, - "search": search or {"type": "vector"}, - "runtime": runtime, - } - }, + "indexes": {"knowledge": binding}, } if schema_overrides is not None: config["indexes"]["knowledge"]["schema_overrides"] = schema_overrides @@ -141,6 +143,56 @@ async def test_server_startup_success(monkeypatch, existing_index, mcp_config_pa await server.shutdown() +@pytest.mark.asyncio +async def test_server_startup_succeeds_for_fulltext_without_vectorizer( + monkeypatch, existing_index, mcp_config_path +): + index = await existing_index( + index_name="mcp-startup-fulltext", + storage_type="hash", + ) + original_build_vectorizer = RedisVLMCPServer._build_vectorizer + build_vectorizer_called = False + + def tracked_build_vectorizer(self): + nonlocal build_vectorizer_called + build_vectorizer_called = True + return original_build_vectorizer(self) + + monkeypatch.setattr( + "redisvl.mcp.server.resolve_vectorizer_class", + lambda class_name: FakeVectorizer, + ) + monkeypatch.setattr( + RedisVLMCPServer, + "_build_vectorizer", + tracked_build_vectorizer, + ) + server = RedisVLMCPServer( + MCPSettings( + config=mcp_config_path( + redis_name=index.name, + search={"type": "fulltext"}, + runtime_overrides={ + "vector_field_name": None, + "default_embed_text_field": None, + }, + include_vectorizer=False, + ) + ) + ) + + await server.startup() + + started_index = await server.get_index() + assert await started_index.exists() is True + assert build_vectorizer_called is False + with pytest.raises(RuntimeError, match="vectorizer is not configured"): + await server.get_vectorizer() + + await server.shutdown() + + @pytest.mark.asyncio async def test_server_fails_when_hybrid_config_requires_native_runtime( monkeypatch, existing_index, mcp_config_path, async_client diff --git a/tests/integration/test_mcp/test_upsert_tool.py b/tests/integration/test_mcp/test_upsert_tool.py index d6736007..6711790b 100644 --- a/tests/integration/test_mcp/test_upsert_tool.py +++ b/tests/integration/test_mcp/test_upsert_tool.py @@ -94,13 +94,39 @@ async def upsertable_index(async_client, worker_id): await index.delete(drop=True) +@pytest.fixture +async def fulltext_only_upsert_index(async_client, worker_id): + schema = IndexSchema.from_dict( + { + "index": { + "name": f"mcp-upsert-fulltext-{worker_id}", + "prefix": f"mcp-upsert-fulltext:{worker_id}", + "storage_type": "hash", + }, + "fields": [ + {"name": "content", "type": "text"}, + {"name": "category", "type": "tag"}, + {"name": "rating", "type": "numeric"}, + ], + } + ) + index = AsyncSearchIndex(schema=schema, redis_client=async_client) + await index.create(overwrite=True, drop=True) + + yield index + + await index.delete(drop=True) + + @pytest.fixture def mcp_config_path(tmp_path: Path, redis_url: str): def factory( *, redis_name: str, read_only: bool = False, + search: dict | None = None, runtime_overrides: dict[str, Any] | None = None, + include_vectorizer: bool = True, ) -> str: runtime = { "text_field_name": "content", @@ -114,20 +140,21 @@ def factory( if runtime_overrides: runtime.update(runtime_overrides) + binding = { + "redis_name": redis_name, + "search": search or {"type": "vector"}, + "runtime": runtime, + } + if include_vectorizer: + binding["vectorizer"] = { + "class": "RecordingVectorizer", + "model": "fake-model", + "dims": 3, + } + config = { "server": {"redis_url": redis_url}, - "indexes": { - "knowledge": { - "redis_name": redis_name, - "vectorizer": { - "class": "RecordingVectorizer", - "model": "fake-model", - "dims": 3, - }, - "search": {"type": "vector"}, - "runtime": runtime, - } - }, + "indexes": {"knowledge": binding}, } config_path = tmp_path / ( f"{redis_name}-{'readonly' if read_only else 'readwrite'}.yaml" @@ -149,15 +176,20 @@ async def started_server(monkeypatch, upsertable_index, mcp_config_path): async def factory( *, + redis_name: str | None = None, read_only: bool = False, + search: dict | None = None, runtime_overrides: dict[str, Any] | None = None, + include_vectorizer: bool = True, ) -> RedisVLMCPServer: server = RedisVLMCPServer( MCPSettings( config=mcp_config_path( - redis_name=upsertable_index.schema.index.name, + redis_name=redis_name or upsertable_index.schema.index.name, read_only=read_only, + search=search, runtime_overrides=runtime_overrides, + include_vectorizer=include_vectorizer, ) ) ) @@ -203,6 +235,54 @@ async def test_upsert_records_inserts_rows_into_hash_index( assert stored["category"] == "science" +@pytest.mark.asyncio +async def test_upsert_records_supports_plain_writes_without_vector_configuration( + started_server, fulltext_only_upsert_index +): + server = await started_server( + redis_name=fulltext_only_upsert_index.schema.index.name, + search={"type": "fulltext"}, + runtime_overrides={ + "vector_field_name": None, + "default_embed_text_field": None, + }, + include_vectorizer=False, + ) + + response = await upsert_records( + server, + records=[{"content": "fulltext upsert", "category": "science", "rating": 5}], + ) + + assert response["status"] == "success" + assert response["keys_upserted"] == 1 + stored = await fulltext_only_upsert_index.fetch( + _record_id_from_key(response["keys"][0]) + ) + assert stored is not None + assert stored["content"] == "fulltext upsert" + assert stored["category"] == "science" + + +@pytest.mark.asyncio +async def test_upsert_records_requires_vectors_when_embedding_is_disabled( + started_server, +): + server = await started_server( + search={"type": "fulltext"}, + runtime_overrides={"default_embed_text_field": None}, + include_vectorizer=False, + ) + + with pytest.raises(RedisVLMCPError, match="embedding") as exc_info: + await upsert_records( + server, + records=[{"content": "vector missing", "category": "science"}], + ) + + assert exc_info.value.code == MCPErrorCode.INVALID_REQUEST + + @pytest.mark.asyncio async def test_upsert_records_updates_existing_row_with_id_field( started_server, upsertable_index diff --git a/tests/unit/test_mcp/test_config.py b/tests/unit/test_mcp/test_config.py index 2398ebad..7631e368 100644 --- a/tests/unit/test_mcp/test_config.py +++ b/tests/unit/test_mcp/test_config.py @@ -177,6 +177,31 @@ def test_mcp_config_binding_helpers(): assert config.redis_name == "docs-index" +def test_vector_search_config_can_omit_text_field_name(): + config = _valid_config() + del config["indexes"]["knowledge"]["runtime"]["text_field_name"] + + loaded = MCPConfig.model_validate(config) + + assert loaded.search.type == "vector" + assert loaded.runtime.text_field_name is None + + +def test_fulltext_config_can_omit_vector_settings_and_vectorizer(): + config = _valid_config() + config["indexes"]["knowledge"]["search"] = {"type": "fulltext"} + del config["indexes"]["knowledge"]["vectorizer"] + del config["indexes"]["knowledge"]["runtime"]["vector_field_name"] + del config["indexes"]["knowledge"]["runtime"]["default_embed_text_field"] + + loaded = MCPConfig.model_validate(config) + + assert loaded.search.type == "fulltext" + assert loaded.vectorizer is None + assert loaded.runtime.vector_field_name is None + assert loaded.runtime.default_embed_text_field is None + + def test_mcp_config_merges_schema_overrides_into_inspection_result(): config_dict = _valid_config() config_dict["indexes"]["knowledge"]["schema_overrides"] = { @@ -267,6 +292,19 @@ def test_mcp_config_validates_runtime_mapping_against_effective_schema(): config.to_index_schema(_inspected_schema()) +def test_fulltext_config_does_not_require_vector_mapping_in_schema(): + config_dict = _valid_config() + config_dict["indexes"]["knowledge"]["search"] = {"type": "fulltext"} + del config_dict["indexes"]["knowledge"]["vectorizer"] + del config_dict["indexes"]["knowledge"]["runtime"]["vector_field_name"] + del config_dict["indexes"]["knowledge"]["runtime"]["default_embed_text_field"] + config = MCPConfig.model_validate(config_dict) + + schema = config.to_index_schema(_inspected_schema()) + + assert isinstance(schema, IndexSchema) + + def test_load_mcp_config_requires_exactly_one_binding(tmp_path: Path): config_path = tmp_path / "mcp.yaml" config_path.write_text( @@ -302,6 +340,34 @@ def test_mcp_config_requires_search_type(): MCPConfig.model_validate(config) +def test_fulltext_config_requires_text_field_name(): + config = _valid_config() + config["indexes"]["knowledge"]["search"] = {"type": "fulltext"} + del config["indexes"]["knowledge"]["runtime"]["text_field_name"] + del config["indexes"]["knowledge"]["vectorizer"] + del config["indexes"]["knowledge"]["runtime"]["vector_field_name"] + del config["indexes"]["knowledge"]["runtime"]["default_embed_text_field"] + + with pytest.raises(ValueError, match="runtime.text_field_name"): + MCPConfig.model_validate(config) + + +def test_hybrid_config_requires_vector_field_and_vectorizer(): + config = _valid_config() + config["indexes"]["knowledge"]["search"] = {"type": "hybrid"} + del config["indexes"]["knowledge"]["runtime"]["vector_field_name"] + + with pytest.raises(ValueError, match="runtime.vector_field_name"): + MCPConfig.model_validate(config) + + config = _valid_config() + config["indexes"]["knowledge"]["search"] = {"type": "hybrid"} + del config["indexes"]["knowledge"]["vectorizer"] + + with pytest.raises(ValueError, match="vectorizer"): + MCPConfig.model_validate(config) + + def test_mcp_config_rejects_invalid_search_type(): config = _valid_config() config["indexes"]["knowledge"]["search"] = {"type": "semantic"} @@ -310,6 +376,26 @@ def test_mcp_config_rejects_invalid_search_type(): MCPConfig.model_validate(config) +def test_server_side_embedding_requires_vector_field_and_vectorizer(): + config = _valid_config() + config["indexes"]["knowledge"]["search"] = {"type": "fulltext"} + del config["indexes"]["knowledge"]["runtime"]["vector_field_name"] + + with pytest.raises( + ValueError, match="default_embed_text_field requires runtime.vector_field_name" + ): + MCPConfig.model_validate(config) + + config = _valid_config() + config["indexes"]["knowledge"]["search"] = {"type": "fulltext"} + del config["indexes"]["knowledge"]["vectorizer"] + + with pytest.raises( + ValueError, match="default_embed_text_field requires vectorizer" + ): + MCPConfig.model_validate(config) + + @pytest.mark.parametrize( ("search_type", "params"), [ diff --git a/tests/unit/test_mcp/test_search_tool_unit.py b/tests/unit/test_mcp/test_search_tool_unit.py index 3e136053..d5a92244 100644 --- a/tests/unit/test_mcp/test_search_tool_unit.py +++ b/tests/unit/test_mcp/test_search_tool_unit.py @@ -46,6 +46,8 @@ def _config_with_search( search_type: str, params: dict[str, Any] | None = None, runtime_overrides: dict[str, Any] | None = None, + *, + include_vectorizer: bool = True, ) -> MCPConfig: runtime_config = { "text_field_name": "content", @@ -57,17 +59,18 @@ def _config_with_search( if runtime_overrides: runtime_config.update(runtime_overrides) + binding = { + "redis_name": "docs-index", + "search": {"type": search_type, "params": params or {}}, + "runtime": runtime_config, + } + if include_vectorizer: + binding["vectorizer"] = {"class": "FakeVectorizer", "model": "test-model"} + return MCPConfig.model_validate( { "server": {"redis_url": "redis://localhost:6379"}, - "indexes": { - "knowledge": { - "redis_name": "docs-index", - "vectorizer": {"class": "FakeVectorizer", "model": "test-model"}, - "search": {"type": search_type, "params": params or {}}, - "runtime": runtime_config, - } - }, + "indexes": {"knowledge": binding}, } ) @@ -94,15 +97,17 @@ def __init__( search_type: str = "vector", search_params: dict[str, Any] | None = None, runtime_overrides: dict[str, Any] | None = None, + include_vectorizer: bool = True, ): self.config = _config_with_search( search_type, search_params, runtime_overrides, + include_vectorizer=include_vectorizer, ) self.mcp_settings = SimpleNamespace(tool_search_description=None) self.index = FakeIndex() - self.vectorizer = FakeVectorizer() + self.vectorizer = FakeVectorizer() if include_vectorizer else None self.registered_tools = [] self.native_hybrid_supported = False @@ -110,6 +115,8 @@ async def get_index(self): return self.index async def get_vectorizer(self): + if self.vectorizer is None: + raise RuntimeError("MCP server vectorizer is not configured") return self.vectorizer async def run_guarded(self, operation_name, awaitable): @@ -328,6 +335,11 @@ async def test_search_records_builds_fulltext_query(monkeypatch): "stopwords": None, "text_weights": {"medical": 2.5}, }, + runtime_overrides={ + "vector_field_name": None, + "default_embed_text_field": None, + }, + include_vectorizer=False, ) built_queries = [] @@ -372,6 +384,38 @@ async def fake_query(query): assert "hybrid_score" not in response["results"][0]["record"] +@pytest.mark.asyncio +async def test_search_records_fulltext_does_not_touch_vectorizer_when_unconfigured( + monkeypatch, +): + server = FakeServer( + search_type="fulltext", + runtime_overrides={ + "vector_field_name": None, + "default_embed_text_field": None, + }, + include_vectorizer=False, + ) + built_queries = [] + + class FakeTextQuery(FakeQuery): + def __init__(self, **kwargs): + built_queries.append(kwargs) + super().__init__(**kwargs) + + async def fake_query(query): + server.index.query_calls.append(query) + return [{"id": "doc:2", "__score": "1.5", "content": "medical science"}] + + monkeypatch.setattr("redisvl.mcp.tools.search.TextQuery", FakeTextQuery) + server.index.query = fake_query + + response = await search_records(server, query="medical science") + + assert built_queries[0]["return_fields"] == ["content", "category", "rating"] + assert response["results"][0]["record"]["content"] == "medical science" + + @pytest.mark.asyncio async def test_search_records_builds_hybrid_query_for_native_runtime(monkeypatch): server = FakeServer( diff --git a/tests/unit/test_mcp/test_server.py b/tests/unit/test_mcp/test_server.py index 9adea52b..e88b8a38 100644 --- a/tests/unit/test_mcp/test_server.py +++ b/tests/unit/test_mcp/test_server.py @@ -42,6 +42,7 @@ def _startup_config(): runtime=SimpleNamespace(max_concurrency=1, startup_timeout_seconds=1), server=SimpleNamespace(redis_url="redis://localhost:6379"), redis_name="idx", + requires_startup_vectorizer=True, vectorizer=SimpleNamespace( class_name="FakeVectorizer", to_init_kwargs=lambda: {}, diff --git a/tests/unit/test_mcp/test_upsert_tool_unit.py b/tests/unit/test_mcp/test_upsert_tool_unit.py index b8f6e95d..15d3d5ca 100644 --- a/tests/unit/test_mcp/test_upsert_tool_unit.py +++ b/tests/unit/test_mcp/test_upsert_tool_unit.py @@ -11,7 +11,27 @@ from redisvl.schema import IndexSchema -def _schema(storage_type: str = "hash") -> IndexSchema: +def _schema( + storage_type: str = "hash", *, include_vector_field: bool = True +) -> IndexSchema: + fields: list[dict[str, Any]] = [ + {"name": "content", "type": "text"}, + {"name": "category", "type": "tag"}, + ] + if include_vector_field: + fields.append( + { + "name": "embedding", + "type": "vector", + "attrs": { + "algorithm": "flat", + "dims": 3, + "distance_metric": "cosine", + "datatype": "float32", + }, + } + ) + return IndexSchema.from_dict( { "index": { @@ -19,20 +39,7 @@ def _schema(storage_type: str = "hash") -> IndexSchema: "prefix": "doc", "storage_type": storage_type, }, - "fields": [ - {"name": "content", "type": "text"}, - {"name": "category", "type": "tag"}, - { - "name": "embedding", - "type": "vector", - "attrs": { - "algorithm": "flat", - "dims": 3, - "distance_metric": "cosine", - "datatype": "float32", - }, - }, - ], + "fields": fields, } ) @@ -40,28 +47,37 @@ def _schema(storage_type: str = "hash") -> IndexSchema: def _config( storage_type: str = "hash", *, + search_type: str = "vector", + include_vector_field: bool = True, + include_vectorizer: bool = True, + include_embed_source: bool = True, max_upsert_records: int = 5, skip_embedding_if_present: bool = True, ) -> MCPConfig: + runtime = { + "text_field_name": "content", + "default_limit": 2, + "max_limit": 5, + "max_upsert_records": max_upsert_records, + "skip_embedding_if_present": skip_embedding_if_present, + } + if include_vector_field: + runtime["vector_field_name"] = "embedding" + if include_embed_source: + runtime["default_embed_text_field"] = "content" + + binding = { + "redis_name": "docs-index", + "search": {"type": search_type}, + "runtime": runtime, + } + if include_vectorizer: + binding["vectorizer"] = {"class": "FakeVectorizer", "model": "test-model"} + return MCPConfig.model_validate( { "server": {"redis_url": "redis://localhost:6379"}, - "indexes": { - "knowledge": { - "redis_name": "docs-index", - "vectorizer": {"class": "FakeVectorizer", "model": "test-model"}, - "search": {"type": "vector"}, - "runtime": { - "text_field_name": "content", - "vector_field_name": "embedding", - "default_embed_text_field": "content", - "default_limit": 2, - "max_limit": 5, - "max_upsert_records": max_upsert_records, - "skip_embedding_if_present": skip_embedding_if_present, - }, - } - }, + "indexes": {"knowledge": binding}, } ) @@ -99,8 +115,10 @@ async def aembed_many(self, contents: list[str], **kwargs): class FakeIndex: - def __init__(self, storage_type: str = "hash"): - self.schema = _schema(storage_type) + def __init__( + self, storage_type: str = "hash", *, include_vector_field: bool = True + ): + self.schema = _schema(storage_type, include_vector_field=include_vector_field) self.load_calls = [] self.keys_to_return = ["doc:1"] self.load_exception = None @@ -124,24 +142,34 @@ def __init__( self, *, storage_type: str = "hash", + search_type: str = "vector", + include_vector_field: bool = True, + include_vectorizer: bool = True, + include_embed_source: bool = True, max_upsert_records: int = 5, skip_embedding_if_present: bool = True, vectorizer: FakeVectorizer | None = None, ): self.config = _config( storage_type, + search_type=search_type, + include_vector_field=include_vector_field, + include_vectorizer=include_vectorizer, + include_embed_source=include_embed_source, max_upsert_records=max_upsert_records, skip_embedding_if_present=skip_embedding_if_present, ) self.mcp_settings = SimpleNamespace(tool_upsert_description=None) - self.index = FakeIndex(storage_type) - self.vectorizer = vectorizer or FakeVectorizer() + self.index = FakeIndex(storage_type, include_vector_field=include_vector_field) + self.vectorizer = vectorizer or FakeVectorizer() if include_vectorizer else None self.registered_tools = [] async def get_index(self): return self.index async def get_vectorizer(self): + if self.vectorizer is None: + raise RuntimeError("MCP server vectorizer is not configured") return self.vectorizer async def run_guarded(self, operation_name: str, awaitable: Any): @@ -224,6 +252,78 @@ async def test_upsert_records_deep_copies_nested_values_before_loading(): assert records[0]["embedding"] == [0.1, 0.2, 0.3] +@pytest.mark.asyncio +async def test_upsert_records_supports_plain_writes_without_vector_configuration(): + server = FakeServer( + storage_type="hash", + search_type="fulltext", + include_vector_field=False, + include_vectorizer=False, + include_embed_source=False, + ) + + response = await upsert_records( + server, + records=[{"id": "alpha", "content": "alpha doc", "category": "science"}], + id_field="id", + ) + + assert response["keys_upserted"] == 1 + assert server.index.load_calls[0]["data"][0] == { + "id": "alpha", + "content": "alpha doc", + "category": "science", + } + + +@pytest.mark.asyncio +async def test_upsert_records_requires_supplied_vectors_without_server_side_embedding(): + server = FakeServer( + storage_type="hash", + search_type="fulltext", + include_vectorizer=False, + include_embed_source=False, + ) + + with pytest.raises(RedisVLMCPError, match="embedding") as exc_info: + await upsert_records( + server, + records=[{"id": "alpha", "content": "alpha doc"}], + id_field="id", + ) + + assert exc_info.value.code == MCPErrorCode.INVALID_REQUEST + assert server.index.load_calls == [] + + +@pytest.mark.asyncio +async def test_upsert_records_accepts_supplied_vectors_without_server_side_embedding(): + server = FakeServer( + storage_type="hash", + search_type="fulltext", + include_vectorizer=False, + include_embed_source=False, + ) + + response = await upsert_records( + server, + records=[ + { + "id": "alpha", + "content": "alpha doc", + "category": "science", + "embedding": [0.1, 0.2, 0.3], + } + ], + id_field="id", + ) + + assert response["keys_upserted"] == 1 + loaded_record = server.index.load_calls[0]["data"][0] + assert loaded_record["embedding"] == array_to_buffer([0.1, 0.2, 0.3], "float32") + assert server.vectorizer is None + + @pytest.mark.asyncio async def test_upsert_records_rejects_invalid_hash_vector_dimensions_before_serializing(): server = FakeServer(storage_type="hash", skip_embedding_if_present=True)