fix: normalize openai embedding responses

[1zzxy1]

Summary

• normalize openai embedding responses so the provider accepts SDK objects, dict payloads, and JSON string payloads
• keep returning float vectors for both single and batch embedding calls
• add regression tests for object responses, JSON string responses, and empty vectors

Closes #6887

Testing

• uv run --group dev pytest tests/test_openai_embedding_source.py -q

Summary by Sourcery

Normalize OpenAI embedding provider responses to consistently return float vectors across different response formats and add regression coverage.

Bug Fixes:

• Handle OpenAI embedding responses returned as SDK objects, dictionaries, or JSON strings while consistently producing float vectors.
• Raise clear errors when embedding responses are missing data or contain invalid embedding items.

Tests:

• Add regression tests covering SDK object responses, JSON string responses, and empty vector error handling for the OpenAI embedding provider.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the robustness and flexibility of the OpenAI embedding source by standardizing how it processes responses from the OpenAI API. It ensures that the system can gracefully handle various data formats, from native SDK objects to raw JSON strings, and consistently output float vectors, thereby improving reliability and reducing potential integration issues. The addition of dedicated tests provides strong assurance of the new logic's correctness.

Highlights

• Response Normalization: Implemented a new _normalize_embedding_response static method to uniformly process OpenAI embedding responses, accepting SDK objects, dictionary payloads, and JSON string payloads.
• Consistent Output: Ensured that both single and batch embedding calls consistently return float vectors after normalization.
• Robust Error Handling: Added error handling within the normalization process to catch unexpected response types or cases where no vectors are included.
• Comprehensive Testing: Introduced new regression tests to validate the normalization logic across different response formats (SDK objects, JSON strings) and to confirm proper error handling for empty vector responses.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

This pull request introduces a new _normalize_embedding_response method in the OpenAIEmbeddingProvider to robustly handle various response formats (SDK objects, JSON strings) from the OpenAI embedding API. This method ensures proper extraction and type conversion of embedding vectors, with new tests added to validate its behavior, including handling of empty vectors. The review suggests adding null checks within _normalize_embedding_response to prevent AttributeError if the response object or individual item objects within the response's data list are None.

[gemini-code-assist]

The current implementation can raise an AttributeError if response is None, because getattr(None, "data", None) raises an exception. Since the function signature accepts Any, it should be robust against None inputs. You can make this safer by first checking if response is truthy before attempting to access its attributes.

        data = response and (response.get("data") if isinstance(response, dict) else getattr(response, "data", None))

[gemini-code-assist]

Similar to the logic for retrieving data from the response, this part can raise an AttributeError if an item in the data list is None. To prevent this, you should add a check to ensure item is not None before attempting to access its attributes or keys.

            embedding = item and (item.get("embedding") if isinstance(item, dict) else getattr(item, "embedding", None))