Troubleshooting test error

[Nush395]

Setup:
imas_core: 5.5.3
imas: 2.1.0-post1
imas_data_dictionaries: 4.0.0

We're in the process of linking imas-python to imas-core now that it's open sourced. There are several requires_imas tests that now run that previously weren't being run in our CI. One class of errors we're now seeing is:

failed on setup with "imas.exception.DataEntryException: IDS 'equilibrium', occurrence 0 is empty."
requires_imas = None

    @pytest.fixture
    def test_data(requires_imas):
>       db_entry = imas.training.get_training_db_entry()

[test/test_ids_ascii_data](http://test/test_ids_ascii_data).py:22: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
training.py:22: in get_training_db_entry
    ids = entry.get(ids_name, autoconvert=False)
db_entry.py:356: in get
    return self._get(
db_entry.py:568: in _get
    dd_version = self._dbe_impl.read_dd_version(ids_name, occurrence)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <imas.backends.imas_core.db_entry_al.ALDBEntryImpl object at 0x12643ce82db0>
ids_name = 'equilibrium', occurrence = 0

    def read_dd_version(self, ids_name: str, occurrence: int) -> str:
        if self._db_ctx is None:
            raise RuntimeError("Database entry is not open.")
        # Mixing contexts can be problematic, ensure all lazy contexts are closed:
        self._clear_lazy_ctx_cache()
    
        ll_path = ids_name
        if occurrence != 0:
            ll_path += f"/{occurrence}"
    
        datapath = "ids_properties" if self.backend == "uda" else ""
        with self._db_ctx.global_action(ll_path, READ_OP, datapath) as read_ctx:
            time_mode_path = "ids_properties/homogeneous_time"
            time_mode = read_ctx.read_data(time_mode_path, "", INTEGER_DATA, 0)
            dd_version_path = "ids_properties/version_put/data_dictionary"
            dd_version = read_ctx.read_data(dd_version_path, "", CHAR_DATA, 1)
    
        if time_mode not in IDS_TIME_MODES:
>           raise DataEntryException(
                f"IDS {ids_name!r}, occurrence {occurrence} is empty."
            )
E           imas.exception.DataEntryException: IDS 'equilibrium', occurrence 0 is empty.

backends/imas_core/db_entry_al.py:268: DataEntryException

Do you have any advice on what is going wrong here/how to debug?

[maarten-ic]

Hi @Nush395,

The error you're seeing indicates that IMAS-Core doesn't load the test data. The most likely cause for this is that it cannot find the data files.

Could you check the following?

1. Are the *.ids files (two text files, two symlinks) from this folder available in your test environment?
2. If these files are available, does the following code snippet point to the folder where they are stored?

from importlib.resources import files
print(files(imas) / "assets/")

[Nush395]

Hi Martin, the text files exist, e.g. https://github.com/iterorganization/IMAS-Python/blob/develop/imas/test/test_ids_ascii_data.py#L15 passes.

I hadn't noticed that the other files are symlinks, that might not interact nicely with our build system so investigating now. Thanks!

[maarten-ic]

Hi @Nush395,

You're welcome!

If the symlinks are a problem for your build system, then I'd be happy to accept a PR that:

1. Removes the symlinks
2. Renames ITER_134173_106_core_profiles.ids -> core_profiles.ids (and similarly for equilibrium.ids)
3. Updates the path in this test https://github.com/iterorganization/IMAS-Python/blob/develop/imas/test/test_ids_ascii_data.py#L15

The symlinks were there to support an old version of the Access Layer (v4.x) that is not relevant anymore :)

[Nush395]

Hi Maarten, that's good to keep in mind, thanks!

I actually think there is some problem with imas_core here potentially as:

• the core_profiles.ids is loaded without issue
• on inspecting the time mode for the equilibrium.ids it is -999999999

[maarten-ic]

• the core_profiles.ids is loaded without issue

Hmm, that's interesting. Why would it work only for one of the two IDSs? Could it be related to file size? The equilibrium file is 3.6 MB vs 736 KB for the core_profiles.

• on inspecting the time mode for the equilibrium.ids it is -999999999

That's the placeholder value for "there is no value stored" in imas_core. This is incorrect: the first four lines in https://github.com/iterorganization/IMAS-Python/blob/develop/imas/assets/ITER_134173_106_equilibrium.ids indicate that the time mode is 1:

equilibrium/ids_properties/homogeneous_time
	type: 51 (INTEGER_DATA)
	dim: 0
1
[...]

Hope this helps you investigating further!

[Nush395]

Hmm, that's interesting. Why would it work only for one of the two IDSs? Could it be related to file size? The equilibrium file is 3.6 MB vs 736 KB for the core_profiles.

Perhaps? but these are both still very small files so I wouldn't expect loading both to be an issue.

[Nush395]

Ah interesting, then that likely confirms that there is something wrong with my build of imas_core.