Skip to content

GitHub Issue 952: Registry containing MVTC import succeeds with error message #7503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
XingY merged 5 commits into from
Mar 23, 2026
+33 −7
Merged

GitHub Issue 952: Registry containing MVTC import succeeds with error message #7503

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
45589af
GitHub Issue 952: Domain containing MVTC import succeeds with error m…
XingY Mar 18, 2026
5d244d8
crlf
XingY Mar 18, 2026
fe1208e
Merge remote-tracking branch 'origin/develop' into fb_issue952
XingY Mar 23, 2026
b03f899
code review changes
XingY Mar 23, 2026
22617c3
csrf
XingY Mar 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion api/src/org/labkey/api/dataiterator/MapDataIterator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import org.apache.commons.lang3.StringUtils;
import org.apache.logging.log4j.Logger;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.labkey.api.collections.ArrayListMap;
import org.labkey.api.collections.CaseInsensitiveHashMap;
import org.labkey.api.collections.CaseInsensitiveTreeSet;
Expand All @@ -43,7 +44,24 @@ public interface MapDataIterator extends DataIterator
Logger LOGGER = LogHelper.getLogger(MapDataIterator.class, "DataIterators backed by Maps");

boolean supportsGetMap();
Map<String,Object> getMap();
@Nullable Map<String,Object> getMap();
default @Nullable Map<String, Object> getMapExcludeExistingRecord()
{
Map<String, Object> row = getMap();
if (null == row)
return null;

if (!row.containsKey(ExistingRecordDataIterator.EXISTING_RECORD_COLUMN_NAME))
return row;

Map<String, Object> rowClean = new CaseInsensitiveHashMap<>(row.size());
row.forEach((k, v) -> {
if (!ExistingRecordDataIterator.EXISTING_RECORD_COLUMN_NAME.equals(k))
rowClean.put(k, v);
});

return rowClean;
}

/**
* wrap an existing DataIterator to add MapDataIterator interface
Expand Down
11 changes: 11 additions & 0 deletions api/src/org/labkey/api/query/AbstractQueryImportAction.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
import org.labkey.api.action.SpringActionController;
import org.labkey.api.attachments.FileAttachmentFile;
import org.labkey.api.audit.TransactionAuditProvider;
import org.labkey.api.collections.LabKeyCollectors;
import org.labkey.api.data.Container;
import org.labkey.api.data.DbSchema;
import org.labkey.api.data.DbScope;
Expand All @@ -56,6 +57,7 @@
import org.labkey.api.util.CPUTimer;
import org.labkey.api.util.FileStream;
import org.labkey.api.util.FileUtil;
import org.labkey.api.util.JsonUtil;
import org.labkey.api.util.PageFlowUtil;
import org.labkey.api.util.Pair;
import org.labkey.api.util.Path;
Expand All @@ -76,6 +78,7 @@
import java.io.IOException;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
Expand Down Expand Up @@ -761,6 +764,14 @@ protected JSONObject createSuccessResponse(int rowCount)
return response;
}

public static JSONArray prepareRowsResponse(@NotNull Collection<Map<String, Object>> rows)
{
return rows.stream()
.map(JsonUtil::toMapPreserveNonFinite)
.map(JsonUtil::toJsonPreserveNulls)
.collect(LabKeyCollectors.toJSONArray());
}

@Override
protected ApiResponseWriter createResponseWriter() throws IOException
{
Expand Down
2 changes: 1 addition & 1 deletion api/src/org/labkey/api/query/AbstractQueryUpdateService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -465,7 +465,7 @@ public boolean next() throws BatchValidationException
{
boolean ret = super.next();
if (ret)
rows.add(((MapDataIterator)_delegate).getMap());
rows.add(((MapDataIterator)_delegate).getMapExcludeExistingRecord());
Copy link
Contributor

@labkey-nicka labkey-nicka Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will someone know when they should call getMap() versus getMapExcludeExistingRecord()?

Copy link
Contributor Author

@XingY XingY Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Best I can say is it needs to be evaluated case-by-case. In this case, the map is used to prepare response.rows. Otherwise, getMap with full map should be used while still inside the dataiterator.

return ret;
}
};
Expand Down
2 changes: 1 addition & 1 deletion experiment/src/org/labkey/experiment/api/SampleTypeUpdateServiceDI.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ else if (nameObj instanceof Number)
}
}
else
rows.add(((MapDataIterator) _delegate).getMap());
rows.add(((MapDataIterator) _delegate).getMapExcludeExistingRecord());
}
return ret;
}
Expand Down
5 changes: 1 addition & 4 deletions query/src/org/labkey/query/controllers/QueryController.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4739,10 +4739,7 @@ protected JSONObject executeJson(JSONObject json, CommandType commandType, boole
}
else if (commandType != CommandType.importRows)
{
response.put("rows", responseRows.stream()
.map(JsonUtil::toMapPreserveNonFinite)
.map(JsonUtil::toJsonPreserveNulls)
.collect(LabKeyCollectors.toJSONArray()));
response.put("rows", AbstractQueryImportAction.prepareRowsResponse(responseRows));
}

// if there is any provenance information, save it here
Expand Down
Loading