Gluten and Velox LP on Cobalt 100

[pareenaverma]

Gluten and Velox LP on Cobalt 100

[pareenaverma]

Team has been actively working on building both Gluten and Velox on Arm64. The build process is heavy and iterative, it typically takes 60–90 minutes per run and often requires several retries to complete successfully.

Currently, even after successful builds, some components are not fully functional on our Cobalt 100 VM with Ubuntu 24.04. We have raised this issue on the official Gluten repo: Issue #11633, and there’s an existing related issue: Issue #11639.
We are continuing to work on stabilizing the build. Once there a properly working setup, we will raise the PR for review. In parallel, we will validate the steps provided from your side for TPC-DS benchmarking.

We are following the below the steps:

1.Update system and configure 32GB swap (mandatory for Velox)
2.Install build dependencies (git, cmake, ninja, JDK17, etc.)
3. Install Spark 4.1 and set environment variables
4. Build Gluten with Velox backend for Arm64 (./dev/buildbundle-veloxbe.sh)
5. Validate the build (ls package/target | grep velox).

Will share updates as soon as the build is fully functional and validated.
In parallel, I will review and try the steps provided by your side on the Cobalt 100 VM to validate TPC-DS benchmarking and raise a PR.