4.1.0 is broken, unified_http_client.py:225 raises "AttributeError: module 'urllib3' has no attribute 'BaseHTTPResponse'. Did you mean: 'HTTPResponse'?"

[ottovis]

Since 4.1.0 was released a couple of hours ago, (atleast) dbt seed on Databricks in Jobs is broken, raising AttributeError: module 'urllib3' has no attribute 'BaseHTTPResponse'. Did you mean: 'HTTPResponse from line 225, in UnifiedHttpClient ) -> Generator[urllib3.BaseHTTPResponse, None, None]:

I would suggest janking the update from pip, as this seems to break every single cluster that launches since this issue was created

[rbsat]

If you are also using the dbt databricks bundle template, as a quick workaround, you can adjust your resources job yml to pin urllib3 as such:

          libraries:
            - pypi:
                package: dbt-databricks>=1.8.0,<2.0.0
            - pypi:
                package: urllib3>=2.0.0

[grigsos]

If you are also using the dbt databricks bundle template, as a quick workaround, you can adjust your yml to pin urllib3 as such:

      libraries:
        - pypi:
            package: dbt-databricks>=1.8.0,<2.0.0
        - pypi:
            package: urllib3>=2.0.0

Can verify that it fixes the issue

[grigsos]

When digging through log4j-active.log of cluster initialization for the version that was broken, my team has found this line:

Requirement already satisfied: urllib3>=1.26 in /databricks/python/lib/python3.11/site-packages (from databricks-sql-connector[pyarrow]<5.0.0,>=4.0.0->dbt-databricks<2.0.0,>=1.8.0) (1.26.16)

So we can see that unsupported version of urllib3 was being used, in this case 1.26.16.

We are thinking that it's most likely the requests-kerberos that was added today that was causing issues

[vikrantpuppala]

Update: we've just yanked the release from pypi to mitigate the issue. Apologies for the inconvenience caused.

[vikrantpuppala]

Fixed by #678