Description
When using a data source that utilizes the pager parameter for paginated API responses, the retrieve_http function raises an UnboundLocalError because content_compression_method is only assigned in the non-pager branch.
Version
ingestify: 0.14.1
Error
File "/home/.../site-packages/ingestify/infra/fetch/http.py", line 161, in retrieve_http
content_compression_method=content_compression_method,
^^^^^^^^^^^^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'content_compression_method' where it is not associated with a value
Root cause
if pager:
# ... pager logic ...
content_bytes = json.dumps({data_path: data}).encode("utf-8")
# ...
stream = BufferedStream.from_stream(BytesIO(content_bytes))
content_length = len(content_bytes)
# ❌ content_compression_method is NOT set here
else:
# ... streaming logic ...
content_compression_method = detect_compression(raw_stream) # ✅ set here
# ...
return DraftFile(
# ...
content_compression_method=content_compression_method, # 💥 UnboundLocalError when pager=True
# ...
)
Description
When using a data source that utilizes the pager parameter for paginated API responses, the retrieve_http function raises an UnboundLocalError because content_compression_method is only assigned in the non-pager branch.
Version
ingestify: 0.14.1
Error
Root cause