feat: Add priority field to Flavor

[googs1025]

What would you like to be added:

apiVersion: llmaz.io/v1alpha1
kind: OpenModel
metadata:
  name: opt-125m
spec:
  familyName: opt
  source:
    modelHub:
      modelID: facebook/opt-125m
  inferenceConfig:
    flavors:
      - name: h800
        priority: 5  # higher priority
        nodeSelector:
          karpenter.k8s.aws/instance-gpu-name: h800
        limits:
          nvidia.com/gpu: 4
      - name: h100
        priority: 4
        nodeSelector:
          karpenter.k8s.aws/instance-gpu-name: h100
        limits:
          nvidia.com/gpu: 4
      - name: a100
        priority: 3
        nodeSelector:
          karpenter.k8s.aws/instance-gpu-name: a100
        limits:
          nvidia.com/gpu: 4
      - name: a20
        priority: 2
        nodeSelector:
          karpenter.k8s.aws/instance-gpu-name: a20
        limits:
          nvidia.com/gpu: 4
      - name: t4
        priority: 1  # lower priority
        nodeSelector:
          karpenter.k8s.aws/instance-gpu-name: t4
        limits:
          nvidia.com/gpu: 4

Why is this needed:

When multiple flavors are defined for a model, there is currently no explicit way to control their matching order during scheduling. The scheduler uses the order defined in the list, which may not reflect the intended preference.

https://github.com/InftyAI/scheduler-plugins/blob/685a4d9f8a769f7f5634a6680f374a05c72823cd/pkg/plugins/resource_fungibility/resource_fungibility.go#L228-L248

Completion requirements:

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.

[googs1025]

cc @kerthcet

[googs1025]

/assign

[kerthcet]

Thanks @googs1025 , yes, use priority makes it more clear. However, I believe it's simple enough today following the slice order because the flavors are only used by one model respectively, doesn't like the Pod priority, it's sharing across all the Pods, so define the Pod priority is important.

But I'm happy to hear more advices in the future. Let's just pending this now. BTW, once we instroduced the priority field, we need to modify the codes in karpenter and schesuler plugins as well.

[googs1025]

Yes, when I looked at the api, I saw that it was already in the comment. So it's not that important.

// InferenceConfig represents the inference configurations for the model.
type InferenceConfig struct {
	// Flavors represents the accelerator requirements to serve the model.
	// Flavors are fungible following the priority represented by the slice order.
	// +kubebuilder:validation:MaxItems=8
	// +optional
	Flavors []Flavor `json:"flavors,omitempty"`
}