Conversation
- Remove log file from snapshot check, as it contains timestamps and
will always fail
- Same as last commit, remove log and biom files from snapshot check
(they are not stable)
Improve humann tests
- Add missing schema to allOf - Run 'nf-core pipelines lint --fix files_unchanged' to fix linting
|
|
Well, at least it doesn't seem to be failing due to lack of space anymore... but I am a bit flummoxed as to why those snapshots (presumably) match for the docker test but not for the singularity one 🤔 |
|
@miraep8 I suspect what might be going on is that for some of these outputs, 9/10 times the tool generates a byte-per-byte identical output, but every so often a quirk happens that changes a single character or byte and changes the checksum. Perhaps we could check for file presence rather than checksum for these particular files? |
Files generated by mifaser can sometimes not be byte-by-byte identical, failing snapshot checksum assertions. This changes makes it so test assertions check only filename and existence. This should probably be fixed once mifaser is added to nf-core/modules. See #35 for discussion.
Temporary solution until these modules (humann, mifaser) are added to nf-core/modules and their tests are sorted.
|
@miraep8 you probably already knew that, but separate issue is that sometimes singularity pull is timing out > doesn't generate outputs > they are missing from the snapshot. |
Respect the template of ./nextflow.config
|
Hmm, I think we've got two separate issues here: singularity humann is returning empty files (with the hash d41d8cd98f00b204e9800998ecf8427e) , and it looks like we have something funny with mifaser as well. I don't think we can just check file presence absence until we understand the differences a bit better. |
|
There are some snapshot empty files as well, which we will need to fix. |
Oh some, maybe all, of these are stub outputs so thats expected |
Just to confirm, this is happening even when the HUMAnN image is pulled successfully? I thought this was only when the singularity image pull timed out. The fact that both of the errors are happening in local modules makes me think that perhaps we should try and prioritise adding them to nf-core/modules? That way these problems can be investigated more closely, and separated from the scope of the pipeline. |
|
@nickp60 in face of the recent discussions that HUMAnN and mifaser may be generating incorrect/empty files, I may revert some of the commits I did here, if that's okay. |
PR checklist
nf-core pipelines lint).nextflow run . -profile test,docker --outdir <OUTDIR>).nextflow run . -profile debug,test,docker --outdir <OUTDIR>).docs/usage.mdis updated.docs/output.mdis updated.CHANGELOG.mdis updated.README.mdis updated (including new tool citations and authors/contributors).