WIP: Refactor Parallel Graph Algorithms to Use a Centralized Parallel Configuration with Flexible Iterators

[dPys]

Summary:

This PR introduces a new execute_parallel function that simplifies algorithm parallelization logic by obviating the need for separate joblib calls for each algorithm while enabling much greater flexibility (for developers and the user).

Key Updates:

1. Add execute_parallel:

   • Centralizes logic for parallel execution across data chunks.
   • Uses iterator_func to customize how data (nodes, edges, etc.) is iterated over.
   • Supports both default and custom chunking via get_chunks.

2. Remove create_iterables:

   • Simplified by moving to iterator_func within execute_parallel.

3. Thread-safe Joblib config:

   • Added a parallel_config context manager that uses thread-local storage to manage Joblib settings (like backend/verbose) without interference during concurrent runs.

4. Refactor betweenness_centrality & edge_betweenness_centrality to use execute_parallel as a POC.

Why This Matters:

• Parsimony: Reduces code complexity by consolidating parallel execution logic into one function, making the codebase cleaner and easier to maintain.
• Flexibility: execute_parallel and iterator_func make this adaptable for any graph algs that need parallelism.
• DRY: Centralized parallel logic reduces duplication.
• Maintainability: Easier to extend and manage without create_iterables.

TODO:

• Add unit test for execute_parallel once the interface gets the green-light
• Perform regression test to ensure equivalent functionality
• Finish refactoring the other algorithms to incorporate execute_parallel in place of existing separate joblib calls

[dschult]

These changes to the centralized processing of chunks seem good. I like the general iterator_func capability, and it still allows inclusion of common cases like "nodes".

So +1 on the the interface from me, though I'd like to hear from @Schefflera-Arboricola too. :)

Thanks for this!

[dschult]

This looks good to me!
Sorry for the delay in review.

[dPys]

No problem! Looks like we still need @Schefflera-Arboricola to also approve before this can get merged.