ModularPipeline.save_pretrained: change the default overwrite_modular_index to be True

[yiyixuxu]

cc @DN6

Just a proposal, but I think since save_pretrained now saves the loaded components by default, it makes sense for overwrite_modular_index to default to True as well — otherwise the result is not very intuitive: e.g. when you run

  pipe = ModularPipeline.from_pretrained(repo_id)
  pipe.load_components() 
  pipe.save_pretrained(local_dir)

the components are all saved into local_dir, but its modular_model_index.json still points at repo_id — so reloading from the local_dir still fetch everything from hub even though weights are in the local folder

  pipe = ModularPipeline.from_pretrained(local_dir)
  pipe.load_components()

It's especially confusing if you intended to save a self-contained local copy with some custom components

See related issues: #14251, #14640

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[DN6]

@yiyixuxu Sounds good to me. Did you also want to overwrite for cases where local_dir is provided?

pipe = ModularPipeline.from_pretrained(repo_id, local_dir=<path to dir>)
pipe.load_components()

[yiyixuxu]

@DN6
ohh I would not worry about it -> I think ModularPipeline.from_pretrained(repo_id, local_dir=<path to dir>) just download the remote code, not the ndex and weights, no?

Did you also want to overwrite for cases where local_dir is provided?

[stevhliu]

thanks!

[sayakpaul]

Components without a recorded source, such as models added with update_components, are still rewritten to point to the destination.

If we don't call push_to_hub=True and just save locally, what is the expected behaviour here? As per the changes in this PR, the modular index then records the local path?

If so, is there a possibility where users would just upload that and the modular index still points to a local destination? So, maybe we could further clarify a bit:

For a local save, the destination recorded in modular_model_index.json is save_directory. Don't upload that directory separately because the index continues to point to the local path. To publish the saved components, pass push_to_hub=True and repo_id to [~ModularPipeline.save_pretrained]; the index then points to the Hub repository instead.

[sayakpaul]

But is there any reason to include revision here?

[yiyixuxu]

just in case the original entry on modular_model_index has a revision and we overwrite, we need to set it to None since it would not apply for the "local compnent"

[sayakpaul]

Very nice!