Skip to content

Enable max label per anchor in data splitter#589

Open
yliu2-sc wants to merge 1 commit intomainfrom
yliu2/max_label_per_anchor
Open

Enable max label per anchor in data splitter#589
yliu2-sc wants to merge 1 commit intomainfrom
yliu2/max_label_per_anchor

Conversation

@yliu2-sc
Copy link
Copy Markdown
Collaborator

Scope of work done

In previous experiments, for non user defined labels where labels are generated from 1 hop, we need to enable limiting num max labels per anchor to reduce blow up in memory.

This PR updates to include the option through configs to limit max mum labels per anchor.

Where is the documentation for this feature?: N/A

Did you add automated tests or write a test plan?

Updated Changelog.md? NO

Ready for code review?: NO

Comment on lines +40 to +50
def validate_max_labels_per_anchor_node(
max_labels_per_anchor_node: Optional[int],
) -> Optional[int]:
"""Validate the optional per-anchor label cap."""
if max_labels_per_anchor_node is None:
return None
if max_labels_per_anchor_node <= 0:
raise ValueError(
"max_labels_per_anchor_node must be a positive integer when provided."
)
return max_labels_per_anchor_node
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this provides a ton of value (and returning from a "check" seems weird, It should probably just always return none and throw an error fi there's a problem?)

Comment on lines +693 to +695
max_labels_per_anchor_node = validate_max_labels_per_anchor_node(
max_labels_per_anchor_node
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're probably calling this function too much, it really only need to be called here right?

I'm not sure it's super useful to call it everywhere when this is the only place it actually gets used, and otherwise it makes our code more complicated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants