Skip to content

Consolidate checks for completion file exists #271

@TimidRobot

Description

@TimidRobot

Description

The existing shared function used by process scripts:

def check_for_data_files(args, file_paths, QUARTER):
if args.force:
return
for path in file_paths:
if os.path.exists(path):
raise QuantifyingException(
f"Processed data already exists for {QUARTER}", 0
)

and the report functions, for example:

def check_report_completion(args):
""" "
The function checks for the last plot and image
caption created in this script. This helps to
immediately know if all plots in the script have
been created and should not be regenerated.
"""
if args.force:
return
last_entry = shared.path_join(PATHS["data_phase"], "gcs_free_culture.png")
if os.path.exists(last_entry):
raise shared.QuantifyingException(
f"{last_entry} already exists. Report script completed", 0
)

can be combined into a single shared function that serves both needs.

  • rename shared function to something like check_completion_file_exists instead of check_for_data_files
  • add a docstring
  • update message to be more generic
  • remember you can use args.quarter instead of a dedicated QUARTER argument
  • remember file_paths list can contain a single path

Implementation

  • I would be interested in implementing this feature.

Metadata

Metadata

Assignees

Projects

Status

Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions