-
Notifications
You must be signed in to change notification settings - Fork 46
TransferClientV2 #1370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
TransferClientV2 #1370
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
026a05d
TransferClientV2, more JSON:API support, and tunnels fixes
aaschaer 41a3cc4
TransferClientV2 testing data and tests
aaschaer c9a1e76
add missing tests
aaschaer 6a6c220
put CreateTunnelData back and add TunnelCreateDocument and TunnelUpda…
aaschaer f1c9082
Typo fixes in changelog
aaschaer 4f55d7d
Fix JSONAPIPaginator required kwargs
aaschaer 2ed754b
refactors from review
aaschaer File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| Added | ||
| ----- | ||
|
|
||
| - Added the experimental ``TransferClientV2`` client class along with helper | ||
| classes ``TunnelCreateDocument`` and ``TunnelUpdateDocument`` (:pr:`NUMBER`) | ||
| - JSON:API iteration and pagination is now supported through | ||
| ``IterableJSONAPIResponse`` and ``JSONAPIPaginator`` (:pr:`NUMBER`) | ||
|
|
||
| Changed | ||
| ------- | ||
|
|
||
| - All tunnels support in ``TransferClient`` has been labeled as Beta as the | ||
| underlying API is still in development. For the most up to date interfaces | ||
| it is recommended to use the ``TransferClientV2`` (:pr:`NUMBER`) | ||
|
|
||
| Fixed | ||
| ----- | ||
|
|
||
| - Various fixes for tunnels support in ``TransferClient`` (:pr:`NUMBER`) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| import importlib | ||
| import sys | ||
| import typing as t | ||
|
|
||
| __all__ = ( | ||
| "TransferClientV2", | ||
| "TunnelCreateDocument", | ||
| "TunnelUpdateDocument", | ||
| ) | ||
|
|
||
|
|
||
| # imports from `globus_sdk.experimental.transfer_v2` are done lazily to ensure | ||
| # that we do not eagerly import `requests` when attempting to use SDK | ||
| # components which do not need it | ||
| if t.TYPE_CHECKING: | ||
| from .transfer_v2 import ( | ||
| TransferClientV2, | ||
| TunnelCreateDocument, | ||
| TunnelUpdateDocument, | ||
| ) | ||
| else: | ||
| _LAZY_IMPORT_TABLE = { | ||
| "transfer_v2": { | ||
| "TransferClientV2", | ||
| "TunnelCreateDocument", | ||
| "TunnelUpdateDocument", | ||
| } | ||
| } | ||
|
|
||
| def __getattr__(name: str) -> t.Any: | ||
| for modname, items in _LAZY_IMPORT_TABLE.items(): | ||
| if name in items: | ||
| mod = importlib.import_module("." + modname, __name__) | ||
| value = getattr(mod, name) | ||
| setattr(sys.modules[__name__], name, value) | ||
| return value | ||
|
|
||
| raise AttributeError(f"module {__name__} has no attribute {name}") | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| from .client import TransferClientV2 | ||
| from .data import TunnelCreateDocument, TunnelUpdateDocument | ||
|
|
||
| __all__ = ( | ||
| "TransferClientV2", | ||
| "TunnelCreateDocument", | ||
| "TunnelUpdateDocument", | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,292 @@ | ||
| from __future__ import annotations | ||
|
|
||
| import logging | ||
| import typing as t | ||
| import uuid | ||
|
|
||
| from globus_sdk import client, paging, response | ||
| from globus_sdk.response import IterableJSONAPIResponse | ||
| from globus_sdk.scopes import TransferScopes | ||
| from globus_sdk.services.transfer.errors import TransferAPIError | ||
| from globus_sdk.transport import RetryConfig | ||
|
|
||
| from .data import TunnelCreateDocument, TunnelUpdateDocument | ||
| from .transport import TRANSFER_V2_DEFAULT_RETRY_CHECKS | ||
|
|
||
| log = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class TransferClientV2(client.BaseClient): | ||
| r""" | ||
| Client for the `/v2/ routes of the Globus Transfer API | ||
| <https://docs.globus.org/api/transfer/#v2>`_. | ||
|
|
||
| .. sdk-sphinx-copy-params:: BaseClient | ||
|
|
||
| This class provides helper methods for /v2/ functionality in the Transfer | ||
| API as it is developed. | ||
|
|
||
| **Paginated Calls** | ||
|
|
||
| Methods which support pagination can be called as paginated or unpaginated methods. | ||
| If the method name is ``TransferClientV2.foo``, the paginated version is | ||
| ``TransferClientV2.paginated.foo``. | ||
| Using ``TransferClientV2.list_stream_access_points`` as an example:: | ||
|
|
||
| from globus_sdk.experimental import TransferClientV2 | ||
| tc = TransferClientV2(...) | ||
|
|
||
| # this is the unpaginated version | ||
| for stream_access_point in tc.list_stream_access_points(): | ||
| print(stream_access_point["attributes"]["display_name"]) | ||
|
|
||
| # this is the paginated version | ||
| for page in tc.paginated.list_stream_access_points(): | ||
| for stream_access_point in page: | ||
| print(stream_access_point["attributes"]["display_name"]) | ||
|
|
||
| .. automethodlist:: globus_sdk.experimental.TransferClientV2 | ||
| """ | ||
|
|
||
| service_name = "transfer" | ||
| error_class = TransferAPIError | ||
| scopes = TransferScopes | ||
| default_scope_requirements = [TransferScopes.all] | ||
|
|
||
| def _register_standard_retry_checks(self, retry_config: RetryConfig) -> None: | ||
| """Override the default retry checks.""" | ||
| retry_config.checks.register_many_checks(TRANSFER_V2_DEFAULT_RETRY_CHECKS) | ||
|
|
||
| # Tunnel methods | ||
|
|
||
| def create_tunnel( | ||
| self, | ||
| data: dict[str, t.Any] | TunnelCreateDocument, | ||
| ) -> response.GlobusHTTPResponse: | ||
| """ | ||
| :param data: Parameters for the tunnel creation | ||
|
|
||
| .. tab-set:: | ||
|
|
||
| .. tab-item:: Example Usage | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| tc = globus_sdk.experimental.TrasferClientV2(...) | ||
| data = globus_sdk.experimental.TunnelCreateDocument(...) | ||
| result = tc.create_tunnel(data) | ||
| print(result["data"]["id"]) | ||
|
|
||
| .. tab-item:: API Info | ||
|
|
||
| ``POST /v2/tunnels`` | ||
| """ | ||
| log.debug("TransferClientV2.create_tunnel(...)") | ||
| r = self.post("/v2/tunnels", data=data) | ||
| return r | ||
|
|
||
| def update_tunnel( | ||
| self, | ||
| tunnel_id: str | uuid.UUID, | ||
| update_doc: dict[str, t.Any] | TunnelUpdateDocument, | ||
| ) -> response.GlobusHTTPResponse: | ||
| r""" | ||
| :param tunnel_id: The ID of the Tunnel. | ||
| :param update_doc: The document that will be sent to the patch API. | ||
|
|
||
| .. tab-set:: | ||
|
|
||
| .. tab-item:: Example Usage | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| tc = globus_sdk.experimental.TrasferClientV2(...) | ||
| data = globus_sdk.experimental.TunnelUpdateDocument(...) | ||
| result = tc.update_tunnel(tunnel_id, data) | ||
| print(result["data"]) | ||
|
|
||
| .. tab-item:: API Info | ||
|
|
||
| ``PATCH /v2/tunnels/<tunnel_id>`` | ||
| """ | ||
| log.debug(f"TransferClientV2.update_tunnel({tunnel_id}, {update_doc})") | ||
| r = self.patch(f"/v2/tunnels/{tunnel_id}", data=update_doc) | ||
| return r | ||
|
|
||
| def get_tunnel( | ||
| self, | ||
| tunnel_id: str | uuid.UUID, | ||
| *, | ||
| query_params: dict[str, t.Any] | None = None, | ||
| ) -> response.GlobusHTTPResponse: | ||
| """ | ||
| :param tunnel_id: The ID of the Tunnel which we are fetching details about. | ||
| :param query_params: Any additional parameters will be passed through | ||
| as query params. | ||
|
|
||
| .. tab-set:: | ||
|
|
||
| .. tab-item:: Example Usage | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| tc = globus_sdk.experimental.TrasferClientV2(...) | ||
| result = tc.show_tunnel(tunnel_id) | ||
| print(result["data"]) | ||
|
|
||
| .. tab-item:: API Info | ||
|
|
||
| ``GET /v2/tunnels/<tunnel_id>`` | ||
| """ | ||
| log.debug("TransferClientV2.get_tunnel({tunnel_id}, {query_params})") | ||
| r = self.get(f"/v2/tunnels/{tunnel_id}", query_params=query_params) | ||
| return r | ||
|
|
||
| def delete_tunnel( | ||
| self, | ||
| tunnel_id: str | uuid.UUID, | ||
| ) -> response.GlobusHTTPResponse: | ||
| """ | ||
| :param tunnel_id: The ID of the Tunnel to be deleted. | ||
|
|
||
| This will clean up all data associated with a Tunnel. | ||
| Note that Tunnels must be stopped before they can be deleted. | ||
|
|
||
| .. tab-set:: | ||
|
|
||
| .. tab-item:: Example Usage | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| tc = globus_sdk.experimental.TrasferClientV2(...) | ||
| tc.delete_tunnel(tunnel_id) | ||
|
|
||
| .. tab-item:: API Info | ||
|
|
||
| ``DELETE /v2/tunnels/<tunnel_id>`` | ||
| """ | ||
| log.debug(f"TransferClientV2.delete_tunnel({tunnel_id})") | ||
| r = self.delete(f"/v2/tunnels/{tunnel_id}") | ||
| return r | ||
|
|
||
| def list_tunnels( | ||
| self, | ||
| *, | ||
| query_params: dict[str, t.Any] | None = None, | ||
| ) -> IterableJSONAPIResponse: | ||
| """ | ||
| :param query_params: Any additional parameters will be passed through | ||
| as query params. | ||
|
|
||
| This will list all the Tunnels created by the authorized user. | ||
|
|
||
| .. tab-set:: | ||
|
|
||
| .. tab-item:: Example Usage | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| tc = globus_sdk.experimental.TrasferClientV2(...) | ||
| tc.list_tunnels(tunnel_id) | ||
|
|
||
| .. tab-item:: API Info | ||
|
|
||
| ``GET /v2/tunnels/`` | ||
| """ | ||
| log.debug(f"TransferClientV2.list_tunnels({query_params})") | ||
| r = self.get("/v2/tunnels", query_params=query_params) | ||
| return IterableJSONAPIResponse(r) | ||
|
|
||
| def get_tunnel_events( | ||
| self, | ||
| tunnel_id: str | uuid.UUID, | ||
| *, | ||
| query_params: dict[str, t.Any] | None = None, | ||
| ) -> IterableJSONAPIResponse: | ||
| """ | ||
| :param tunnel_id: The ID of the Tunnel which we are fetching events about. | ||
| :param query_params: Any additional parameters will be passed through | ||
| as query params. | ||
|
|
||
| .. tab-set:: | ||
|
|
||
| .. tab-item:: Example Usage | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| tc = globus_sdk.experimental.TrasferClientV2(...) | ||
| result = tc.get_tunnel_events(tunnel_id) | ||
| print(result["data"]) | ||
|
|
||
| .. tab-item:: API Info | ||
|
|
||
| ``GET /v2/tunnels/<tunnel_id>/events`` | ||
| """ | ||
| log.debug(f"TransferClientV2.get_tunnel_events({tunnel_id}, {query_params})") | ||
| r = self.get(f"/v2/tunnels/{tunnel_id}/events", query_params=query_params) | ||
| return IterableJSONAPIResponse(r) | ||
|
|
||
| # Stream access point methods | ||
|
|
||
| def get_stream_access_point( | ||
| self, | ||
| stream_ap_id: str | uuid.UUID, | ||
| *, | ||
| query_params: dict[str, t.Any] | None = None, | ||
| ) -> response.GlobusHTTPResponse: | ||
| """ | ||
| :param stream_ap_id: The ID of the steaming access point to lookup. | ||
| :param query_params: Any additional parameters will be passed through | ||
| as query params. | ||
|
|
||
| Get a stream access point by id. | ||
|
|
||
| .. tab-set:: | ||
|
|
||
| .. tab-item:: Example Usage | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| tc = globus_sdk.experimental.TrasferClientV2(...) | ||
| tc.get_stream_access_point(stream_ap_id) | ||
|
|
||
| .. tab-item:: API Info | ||
|
|
||
| ``GET /v2/stream_access_points/<stream_ap_id>`` | ||
| """ | ||
| log.debug( | ||
| f"TransferClientV2.get_stream_access_point({stream_ap_id}, {query_params})" | ||
| ) | ||
| r = self.get( | ||
| f"/v2/stream_access_points/{stream_ap_id}", query_params=query_params | ||
| ) | ||
| return r | ||
|
|
||
| @paging.has_paginator(paging.JSONAPIPaginator) | ||
| def list_stream_access_points( | ||
| self, | ||
| *, | ||
| query_params: dict[str, t.Any] | None = None, | ||
| ) -> IterableJSONAPIResponse: | ||
| """ | ||
| :param query_params: Any additional parameters will be passed through | ||
| as query params. | ||
|
|
||
| List stream access points. | ||
|
|
||
| .. tab-set:: | ||
|
|
||
| .. tab-item:: Example Usage | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| tc = globus_sdk.experimental.TrasferClientV2(...) | ||
| tc.list_stream_access_points() | ||
|
|
||
| .. tab-item:: API Info | ||
|
|
||
| ``GET /v2/stream_access_points`` | ||
| """ | ||
| log.debug(f"TransferClientV2.list_stream_access_points({query_params})") | ||
| r = self.get("/v2/stream_access_points", query_params=query_params) | ||
| return IterableJSONAPIResponse(r) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| """ | ||
| Data helper classes for constructing Transfer v2 API documents. All classes should | ||
| be Payload types, so they can be passed seamlessly to | ||
| :class:`TransferClient <globus_sdk.TransferClient>` methods without conversion. | ||
| """ | ||
|
|
||
| from .tunnel_documents import TunnelCreateDocument, TunnelUpdateDocument | ||
|
|
||
| __all__ = ("TunnelCreateDocument", "TunnelUpdateDocument") |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.