[Python] Environment variables for PyArrow not needed anymore

[AlenkaF]

Describe the enhancement requested

With the switch to scikit-build-core backend PyArrow now uses env vars from Arrow C++ and so the usage of pyarrow specific env vars, for example PYARROW_WITH_GANDIVA, is not needed anymore. I think these env vars can be removed from Building PyArrow documentation and also from our CI and dev scripts.

Component(s)

Python

[AlenkaF]

cc @raulcd

[raulcd]

Makes sense, we are still prioritizing them on the PyArrow build:

arrow/python/CMakeLists.txt

Lines 305 to 333 in 3faa8e1

	macro(define_option name description arrow_option)
	set("PYARROW_${name}"
	"AUTO"
	CACHE STRING ${description})
	if("${PYARROW_${name}}" STREQUAL "AUTO")
	# by default, first check if env variable exists, otherwise use Arrow C++ config
	set(env_variable "PYARROW_WITH_${name}")
	if(DEFINED ENV{${env_variable}})
	if($ENV{${env_variable}})
	set("PYARROW_BUILD_${name}" ON)
	else()
	set("PYARROW_BUILD_${name}" OFF)
	endif()
	else()
	if(${arrow_option})
	set("PYARROW_BUILD_${name}" ON)
	else()
	set("PYARROW_BUILD_${name}" OFF)
	endif()
	endif()
	else()
	if("${PYARROW_${name}}")
	set("PYARROW_BUILD_${name}" ON)
	else()
	set("PYARROW_BUILD_${name}" OFF)
	endif()
	endif()
	endmacro()

We could potentially remove them and use the Arrow C++ ones only, which we already do if the env variable is not defined.

[AlenkaF]

Ah, missed that priority code there, thank you for explaining!
In that case I am not against keeping it as is.