[C++][CI][Packaging][FlightPRC] Manual ODBC Windows MSI installer signing

[alinaliBQ]

Describe the enhancement requested

The Apache Arrow Flight SQL ODBC Windows MSI installer needs to be signed in order for Windows defender to allow install.

Acceptance Criteria:

Implement manual code signing process to:

• Sign ODBC Windows MSI installer
• Sign ODBC DLL files that will be installed by MSI installer

Reference comment for this issue: #48934 (comment)

Component(s)

FlightRPC, Packaging

[amoeba]

I've sent a request to ASF Infra requesting access to code signing. I'll update here when that's done and I can continue with this.

[alinaliBQ]

Thanks Bryce!

[alinaliBQ]

Hi @amoeba do you know if Windows MSI signing could be included in Arrow 24.0?

[amoeba]

My request with ASF for access to code signing is still in progress. I just pinged them. Once we get that, I think we can move ahead with a patch to implement manual code signing. I did some research on how hard it would be to make the Windows .DLL and .MSI files reproducible and it looks hard if not impossible.

[alinaliBQ]

Got it, I have changed the issue description to indicate manual code signing process instead. Thanks Bryce for helping out!

[amoeba]

I now have access to Windows code signing. This weekend I'll work on doing a dry-run with the cert and jsign so we can know if it will work at release time. I'm almost set up for macOS too.

[amoeba]

I did a test signing today with jsign to confirm everything from ASF infra is set up. It was just a single jsign command as indicated in #48934 (comment) and it works with SmartScreen as expected.

I think we need a few more things here:

1. Since both the .DLL and the final .MSI need to get signed (two steps), the odbc-msvc job needs to be modified to upload the raw materials for the installer (as many files as needed so cpack will run) and the later steps in that workflow may need to be tweaked.
2. We should investigate static linkage on Windows. The driver uses shared linkage on Win32 and, from my understanding of SmartScreen, we may have to sign all the DLL dependencies (which we don't want to do since each signing costs money). Right now, arrow_flight_sql_odbc.dll is dynamically linked to {arrow,arrow_compute,arrow_flight,arrow_flight_sql}.dll. Can we investigate static linkage (i.e., make arrow_flight_sql_odbc.dll only depend on system deps) @alinaliBQ?
3. Modify 04-binary-download.sh to download the raw WiX installer artifacts instead of the MSI
4. Modify 05-binary-upload.sh to jsign the arrow_flight_sql_odbc.dll inside the WiX installer artifacts dir, build the installer, and jsign the installer. Both the jsign command and WiX are hard for existing release managers to run (jsign requires credentials and WiX is Windows only). We should talk about the best way to integrate this into the release process and whether a separate step should be added.

I did a quick research into reproducible builds for Windows .DLL and .MSI and my initial findings weren't that promising but I think more research could be done here. @raulcd @kou what do you think about having a manual step that requires setup and Windows in the release process?

[kou]

How about using GitHub Actions to build MSI with WiX and signed artifacts?

(Uploading the signed artifacts to GitHub Releases -> Run GitHub Actions via workflow_dispatch -> Build the MSI on GitHub Actions and upload it to GitHub Releases -> Download the MSI from GitHub Releases and sign it)