[SPARK-56938][CONNECT][TESTS] Initialize base session in AddArtifactsHandlerSuite

[sarutak]

What changes were proposed in this pull request?

This PR fixes an issue that all tests in AddArtifactsHandlerSuite fails if the suite runs individually.

$ build/sbt 'testOnly org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite'

...

[info] AddArtifactsHandlerSuite:
[info] - single chunk artifact *** FAILED *** (235 milliseconds)
[info]   io.grpc.StatusRuntimeException: INTERNAL: None.get
[info]   at io.grpc.Status.asRuntimeException(Status.java:532)
[info]   at io.grpc.protobuf.StatusProto.toStatusRuntimeException(StatusProto.java:52)
[info]   at org.apache.spark.sql.connect.utils.ErrorUtils$.org$apache$spark$sql$connect$utils$ErrorUtils$$processErrorCommon(ErrorUtils.scala:335)
[info]   at org.apache.spark.sql.connect.utils.ErrorUtils$$anonfun$handleError$1.applyOrElse(ErrorUtils.scala:407)
[info]   at org.apache.spark.sql.connect.utils.ErrorUtils$$anonfun$handleError$1.applyOrElse(ErrorUtils.scala:405)
[info]   at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35)
[info]   at org.apache.spark.sql.connect.service.SparkConnectAddArtifactsHandler.onNext(SparkConnectAddArtifactsHandler.scala:87)
[info]   at org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite.addSingleChunkArtifact(AddArtifactsHandlerSuite.scala:239)
[info]   at org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite.$anonfun$new$1(AddArtifactsHandlerSuite.scala:294)
[info]   at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
[info]   at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
[info]   at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
[info]   at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
[info]   at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:30)
[info]   at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:41)
[info]   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info]   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:20)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
[info]   at org.apache.spark.SparkTestSuite.withFixture(SparkTestSuite.scala:175)
[info]   at org.apache.spark.SparkTestSuite.withFixture$(SparkTestSuite.scala:169)
[info]   at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:30)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
[info]   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
[info]   at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:30)
[info]   at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
Warning: Unable to serialize throwable of type io.grpc.StatusRuntimeException for TestFailed(Ordinal(0, 3),INTERNAL: None.get,AddArtifactsHandlerSuite,org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite,Some(org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite),single chunk artifact,single chunk artifact,Vector(),Vector(),Some(io.grpc.StatusRuntimeException: INTERNAL: None.get),Some(235),Some(IndentedText(- single chunk artifact,single chunk artifact,0)),Some(SeeStackDepthException),Some(org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite),None,pool-1-thread-1-ScalaTest-running-AddArtifactsHandlerSuite,1779160574268), setting it as NotSerializableWrapperException.
[info]   at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
[info]   at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:30)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
[info]   at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
[info]   at scala.collection.immutable.List.foreach(List.scala:323)
[info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info]   at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
[info]   at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
[info]   at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
[info]   at org.scalatest.Suite.run(Suite.scala:1114)
[info]   at org.scalatest.Suite.run$(Suite.scala:1096)
[info]   at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
[info]   at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
[info]   at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:30)
[info]   at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
[info]   at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info]   at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info]   at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:30)
[info]   at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
[info]   at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
[info]   at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
[info]   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[info]   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[info]   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[info]   at java.base/java.lang.Thread.run(Thread.java:840)

...

The reason is this suite doesn't initialize base session before each test.

Why are the changes needed?

Bug fix.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Confirmed all tests in AddArtifactsHandlerSuite passed.

$ build/sbt 'testOnly org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite'

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

cc @peter-toth

[dongjoon-hyun]

Merged to master/4.x/4.2 for Apache Spark 4.2.0 because Apache Spark 4.2.0 RC1 is planned tomorrow.

• [FYI] Spark 4.2.0 RC1 planned for May 20