CSV import: upload support in the file explorer by amirrza777 · Pull Request #24 · mde-optimiser/mdeo-cloud

amirrza777 · 2026-07-14T18:04:39Z

Closes #21

Depends on #23 (branched from amir/csv-import, since this needs .csv registered as a first-class file type). Only the last commit (a81b3fc) is new to this PR; the rest is #23's history. Marking as draft until #23 merges, at which point I'll rebase this onto main.

What this adds

Previously the only way to get a CSV into a project was creating an empty file through the workbench text editor and typing/pasting into it. This adds:

An "Upload CSV..." action in the file explorer's context menu, both at the project root and on individual folders
OS drag-and-drop support: dropping a .csv file from your computer onto the file tree (root or a specific folder) uploads it there

Both paths go through the same upload helper: read the file's text, create it through the existing file service (the same one file creation already uses), and open it in a new tab. Non-.csv files are rejected with a toast instead of silently uploaded; a name collision shows a clear error instead of overwriting.

Verified

Tested both upload paths (root and nested folder), drag-and-drop onto the tree, rejection of a non-.csv file, and the duplicate-name error, all in the running app.

amirrza777 · 2026-07-14T18:12:45Z

Hello Dear Steffen and Niklas,

Progress update on this one.

What's implemented:

"Upload CSV..." action in the file explorer's right-click menu, available both at the project root and on individual folders
OS drag-and-drop: dropping a .csv file from your computer onto the file tree (root or a specific folder) uploads it there
Both paths share one upload helper: read the file's text, create it through the existing file service, open it in a new tab
Non-.csv files are rejected with a toast instead of silently uploaded
A name collision shows a clear error instead of overwriting

Discoverability note: right now this only lives in the right-click context menu, the same place "Create New CSV" and the other file-creation actions live, so there's no visible upload button in the UI. Happy to add a visible button/icon in the Files panel if that's preferred over a context-menu-only action.

Verified in the running app: upload at project root, upload into a nested folder, drag-and-drop onto the tree, rejecting a non-csv file, and the duplicate-name error.

This is stacked on #23 since it needs .csv registered as a file type, will rebase onto main once that merges.

Will wait for any advises and suggestions.

szschaler · 2026-07-24T14:17:58Z

@amirrza777 one build is failing. Could you check what's going on there, please?

Copilot

Pull request overview

This PR adds first-class CSV upload support in the workbench file explorer (context-menu action + OS drag-and-drop) and wires up the required CSV/model-CSV plugin services and model-language contribution plumbing so uploaded CSVs can participate in the existing language/plugin pipeline.

Changes:

Add “Upload CSV…” and OS drag-and-drop CSV upload paths in the file explorer, backed by a shared upload helper that creates files via Monaco’s file service and opens the created file.
Add CSV and model-CSV services (dockerfiles, compose wiring, nginx routing) and register them as default plugins in dev compose.
Extend the model language to support plugin-contributed import <FORMAT> { ... } blocks (including CSV), and update model data/diagram paths to account for CSV-backed instances.

Reviewed changes

Copilot reviewed 49 out of 55 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
platform/common/src/main/kotlin/com/mdeo/common/model/ApiResult.kt	Formatting-only change (blank line normalization).
platform/backend/build.gradle.kts	Formatting-only change (blank line normalization).
infra/docker/workbench/nginx.conf	Proxy routing for `/plugin/csv/` and `/plugin/model-csv/`.
infra/docker/workbench/Dockerfile	Increases Node memory for the workbench build step.
infra/docker/service-model-csv/Dockerfile	New container build for the model-csv plugin service.
infra/docker/service-csv/Dockerfile	New container build for the csv plugin service.
infra/docker-compose-dev.yaml	Registers csv/model-csv plugins + adds new services in dev compose.
app/tsconfig.build.json	Adds TS project references for new CSV/model-CSV packages/services.
app/packages/workbench/src/data/filesystem/uploadFiles.ts	New shared helper to upload CSV files into the project FS and open the result.
app/packages/workbench/src/components/tree/util.ts	Extends DnD callbacks to support OS file drops.
app/packages/workbench/src/components/tree/TreeItem.vue	Handles OS file drops on folder items and forwards to callback.
app/packages/workbench/src/components/tree/Tree.vue	Handles OS file drops on the tree background and forwards to callback.
app/packages/workbench/src/components/files/FileSystemItem.vue	Adds “Upload CSV…” action + hidden file input for folder context menus.
app/packages/workbench/src/components/files/Files.vue	Adds “Upload CSV…” at root context menu + wires OS drop handling to upload helper.
app/packages/service-model/tsconfig.json	Adds reference to `language-model-csv`.
app/packages/service-model/src/index.ts	Updates model service plugin definition (notably dropping `.m_gen` registration).
app/packages/service-model/src/handler/modelDataHandler.ts	Adds CSV-import path for producing `ModelData` from imported CSV content.
app/packages/service-model/package.json	Adds dependency on `@mdeo/language-model-csv`.
app/packages/service-model-csv/vite.config.ts	Builds the served ESM module for model-csv language provider.
app/packages/service-model-csv/tsconfig.json	New TS config for the model-csv service package.
app/packages/service-model-csv/src/served/language.ts	Serves model-csv plugin provider to the language server.
app/packages/service-model-csv/src/index.ts	New model-csv service: registers contribution plugin for `import CSV { ... }`.
app/packages/service-model-csv/package.json	New package definition for model-csv service.
app/packages/service-csv/vite.config.ts	Builds the served ESM module for csv language provider.
app/packages/service-csv/tsconfig.json	New TS config for the csv service package.
app/packages/service-csv/src/served/language.ts	Serves csv plugin provider to the language server.
app/packages/service-csv/src/index.ts	New csv service: registers `.csv` language plugin.
app/packages/service-csv/package.json	New package definition for csv service.
app/packages/language-model/src/plugin/resolvePlugins.ts	New resolver for model contribution plugins (imports + wrapper rules).
app/packages/language-model/src/plugin/modelContributionPlugin.ts	New contract for model-language contribution plugins.
app/packages/language-model/src/modelPlugin.ts	Builds model grammar dynamically from resolved contribution plugins.
app/packages/language-model/src/index.ts	Exports new model contribution plugin APIs.
app/packages/language-model/src/grammar/modelTypes.ts	Adds `BaseModelImport` + `Model.imports` to support contributed imports.
app/packages/language-model/src/grammar/modelRules.ts	Adds `createModelRule(...)` to include contributed import wrapper rules.
app/packages/language-model/src/features/modelScopeProvider.ts	Broadens class reference scoping to work under contributed import blocks.
app/packages/language-model/src/features/diagram-server/modelMetadataManager.ts	Preserves metadata for synthetic CSV nodes across renders when CSV import present.
app/packages/language-model/src/features/diagram-server/modelGModelFactory.ts	Renders synthetic diagram nodes based on CSV import rows.
app/packages/language-model-csv/tsconfig.json	New TS config for the model CSV contribution package.
app/packages/language-model-csv/src/plugin/modelCsvContributionPlugin.ts	Declares the CSV contribution plugin (serialized grammar + import wrapper info).
app/packages/language-model-csv/src/index.ts	Barrel exports for model-csv language/contribution features.
app/packages/language-model-csv/src/grammar/csvImportTypes.ts	AST shapes for `import CSV { Class from "file.csv" }`.
app/packages/language-model-csv/src/grammar/csvImportRules.ts	Parser rules for the CSV import block content.
app/packages/language-model-csv/src/features/csvImport.ts	Implements CSV->instances/links mapping logic (with warnings).
app/packages/language-model-csv/src/csvPlugin.ts	Standalone model-csv language plugin provider.
app/packages/language-model-csv/package.json	New package definition for `@mdeo/language-model-csv`.
app/packages/language-csv/tsconfig.json	New TS config for the csv language package.
app/packages/language-csv/src/index.ts	Barrel exports for csv language package.
app/packages/language-csv/src/grammar/csvTypes.ts	Minimal AST type for CSV file content.
app/packages/language-csv/src/grammar/csvRules.ts	Minimal grammar for treating a CSV as “any text”.
app/packages/language-csv/src/csvPlugin.ts	Standalone csv language plugin provider.
app/packages/language-csv/package.json	New package definition for `@mdeo/language-csv`.
app/package-lock.json	Adds workspace links and dependency updates for new packages.
.gitignore	Adds `.DS_Store`.

Files not reviewed (1)

app/package-lock.json: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    let lastCreatedUri: Uri | undefined;
+
+    for (const file of csvFiles) {
+        const uri = Uri.joinPath(targetFolderUri, file.name);
+        try {
+            const text = await file.text();
+            await fileService.createFile(uri, VSBuffer.fromString(text));
+            lastCreatedUri = uri;
+        } catch (error) {
+            showError(`Failed to upload ${file.name}`, {
+                description: error instanceof Error ? error.message : undefined
+            });
+        }
+    }
+
+    if (lastCreatedUri == undefined) {
+        return;
+    }
+
+    showSuccess(csvFiles.length === 1 ? `Uploaded ${csvFiles[0]?.name}` : `Uploaded ${csvFiles.length} files`);
+


@@ -125,6 +127,8 @@ services:
        condition: service_healthy
      service-config-mdeo:
        condition: service_healthy
+      service-model-csv:
+        condition: service_healthy
    restart: unless-stopped


+    location ^~ /plugin/csv/ {
+        proxy_pass ${PLUGIN_CSV_SERVICE}/;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+    }


+    const result = importCsvEntries(csvEntries, metamodelClasses);
+
+    const modelData: ModelData = {
+        metamodelPath,
+        instances: result.instances,
+        links: result.links
+    };


+                const csvContent = await this.modelState.languageServices.shared.workspace.FileSystemProvider.readFile(uri);
+                const rows = csvContent.split(/\r?\n/).filter((line: string) => line.trim() !== "").map((line: string) => line.split(","));
+                if (rows.length < 2) continue;
+                const classHierarchy = resolveClassChain(classRef, this.reflection).map((c) => c.name);
+                rows.slice(1).forEach((_row: string[], rowIndex: number) => {
+                    const instanceName = `${classRef.name}_${rowIndex}`;


+const config: ServiceConfig<ModelServices> = {
    ...envConfig,
    plugin: modelServicePlugin,
-    languages: [modelLanguageConfig, generatedModelLanguageConfig]
+    languages: [modelLanguageConfig]
 };


…mport

…nversion

…parsing raw file

…-nullable

…kbench

…ce pattern

…n plugin with import CSV syntax

…er argument

…rtup

… instance issues

…ach Model

…SV block

…mProvider

…lock

…teCsvNodes instead

…class hierarchy

… render

The POST /api/projects/{id}/csv-import endpoint materialized a CSV into a standalone .metamodel/.model file pair using metamodel inference. That workflow is superseded by the import CSV {} syntax, which reads CSV data directly at render/execution time against an existing, manually authored metamodel. Removes CsvImportService, CsvImportRoutes, the now-unused CsvModelInference engine and its tests, the backend's :metamodel dependency (no longer needed), and the unused CSV_IMPORT_FAILED error code.

The import CSV {} syntax was hardcoded directly into the base model grammar. Model now exposes a generic extension point (imports: [BaseModelImport], mirroring Config's sections: [BaseConfigSection]), and CSV is implemented as a real contribution plugin using the same SerializedGrammar-based merging approach as config/config-mdeo: - ModelContributionPlugin now carries a serialized grammar and import metadata instead of raw parser rule objects, so it can cross service boundaries as plain JSON - resolveModelPlugins deserializes and wraps each contributed import in a BaseModelImport-extending rule, and createModelRule builds the root Model rule from whatever imports are actually resolved - language-model-csv now builds a real contribution plugin (external references for Class/ID/STRING/NEWLINE so its serialized grammar doesn't duplicate definitions already provided by the host) - New service-model-csv microservice mirrors service-config-mdeo: registers the CSV contribution with the model language and hosts its standalone generated language service - Deleted the dead mutable-registry scaffolding (registerModelContributionPlugin) that caused the original ESM dual-instance bug, and the orphaned files left over from the previous abandoned attempt - modelScopeProvider's class-reference resolution is now plugin-agnostic (dispatches on the "class" property generically instead of hardcoding CsvClassImport by type) - Fixed a pre-existing bug in modelDataHandler where the CSV file path was passed as a full URI to an API expecting a plain relative path Verified end to end: grammar parses with no errors, live diagram rendering from CSV still works, and the model-data execution pipeline correctly resolves CSV-backed instances.

npm ci in CI failed because these two new packages, added as part of the contribution-plugin refactor, were never registered in the committed lock file.

Users could previously only get a CSV into a project by creating an empty file through the workbench text editor. Adds an "Upload CSV..." action to the file explorer's context menu (both at the project root and on individual folders) and OS drag-and-drop support onto the file tree, both going through the same upload path: read the file's text, create it via the existing file service, and open it in a new tab. Non-.csv files are rejected with a toast; a name collision surfaces a clear error instead of silently overwriting.

- uploadFiles.ts: the success toast used the attempted file count/first file name instead of the actually-successful ones, so a partial failure (one file fails, another succeeds) could report the wrong count or the failed file's name as if it had uploaded. - docker-compose-dev.yaml: workbench's depends_on was missing service-csv (only had service-model-csv), even though nginx and the backend's default plugin list both reference /plugin/csv, risking a 502 on startup before service-csv is healthy. - nginx.conf: /plugin/csv/ was missing the proxy_http_version, forwarded headers, and cross-origin headers every other plugin location has. - service-model: register the generated model language (.m_gen), which the platform still produces (e.g. optimizer solutions) and language-model still supports, but this service no longer wired up. - modelDataHandler: a CSV import present in a .m file was replacing all hand-authored objects/links instead of merging with them. - Moved the quote-aware CSV parser out of language-model-csv into language-shared so language-model's live diagram rendering (modelGModelFactory) uses the same correct parser instead of a naive split(",")/split("\n") that miscounts rows on a quoted comma or newline. Verified live: CSV upload, mixed manual + CSV data, and a quoted CSV value all render correctly.

amirrza777 · 2026-07-24T21:34:26Z

Addressed the new round of Copilot comments:

Upload toast: was using the attempted count/first file name rather than the actually-successful ones, so a partial failure (one file fails, another succeeds) could report the wrong count or claim the failed file uploaded. Now tracks successful uploads specifically.
docker-compose-dev.yaml: workbench's depends_on was missing service-csv (only had service-model-csv), even though nginx and the backend both reference /plugin/csv. Added it.
nginx.conf: /plugin/csv/ was missing the same proxy/cross-origin headers every other plugin location has. Fixed to match.
service-model no longer registered .m_gen, a real regression: the platform still produces .m_gen files and language-model still supports it, just wasn't wired up here anymore. Registered it the same way the classic pipeline does.
modelDataHandler was replacing hand-authored objects/links with CSV-imported ones instead of merging both. Now merges.
The live diagram's CSV row-counting used a naive split(",")/split("\n") that miscounts on a quoted comma or newline. Moved the already-correct quote-aware parser from language-model-csv into language-shared so both the backend import and the live diagram rendering use the same one.

Verified live: CSV upload, mixed manual + CSV data in the same file, and the plugin registrations all work correctly.

szschaler requested a review from Copilot July 24, 2026 14:18

Copilot started reviewing on behalf of szschaler July 24, 2026 14:18 View session

Copilot AI reviewed Jul 24, 2026

View reviewed changes

amirrza777 added 25 commits July 24, 2026 18:29

Fix edge label swap in model-transformation diagrams; add CSV model i…

423dd9c

…mport

Add CSV model import: inference engine, backend endpoint, and unit tests

31aee44

Remove edge label fix from CSV import branch (belongs in separate PR)

4c4cef2

Add CSV import block syntax to model language and CSV-to-ModelData co…

6fb7a28

…nversion

Fix CSV import: use fileDataService to get metamodel data instead of …

56665c2

…parsing raw file

Fix CSV import: use fileDataService to get computed metamodel data

bdeebfa

Fix compile error: handle JsonNull explicitly instead of elvis on non…

6414745

…-nullable

Write model file in DSL text format instead of JSON

53c1666

Fix relative metamodel path in generated model DSL

00d2729

Add language-csv and service-csv packages for CSV file support in wor…

a32c782

…kbench

Fix service-csv Dockerfile and compose config to match existing servi…

e38e4f1

…ce pattern

Remove hardcoded port mapping for service-csv to avoid conflicts

da1b096

Final CSV import: updated routes, inference engine, and tests

bf0359e

Add .DS_Store to gitignore

81cd1e5

Add ModelContributionPlugin system and language-model-csv contributio…

3a29f16

…n plugin with import CSV syntax

Fix contribution plugin grammar merging and remove invalid TokenBuild…

ee36b46

…er argument

Register CSV contribution plugin as built-in to model language at sta…

7f86a35

…rtup

Hardcode CSV contribution directly in modelPlugin to avoid ESM module…

98d67dc

… instance issues

Add import CSV syntax directly to model grammar as a first-class feature

f000b87

Fix scope resolution for CsvClassImport class references

0e5c47f

Fix scope resolution for CsvClassImport: traverse two levels up to re…

df595c6

…ach Model

Register modelDataHandler in service-model for CSV import block support

7d91c17

Read CSV files in GModelFactory to render diagram nodes from import C…

bfb411d

…SV block

Fix modelGModelFactory: add CSV import block rendering with FileSyste…

faa013b

…mProvider

Add createCsvNodes to render diagram nodes directly from CSV import b…

c334ddf

…lock

amirrza777 added 8 commits July 24, 2026 18:29

Remove synthetic PartialObjectInstance from extractElements, use crea…

55a8144

…teCsvNodes instead

Fix createCsvNodes: use proper GObjectNode builder with metadata and …

f2897b3

…class hierarchy

Persist CSV node positions by carrying forward metadata from previous…

31f318b

… render

Add language-model-csv and service-model-csv to package-lock.json

26871d1

npm ci in CI failed because these two new packages, added as part of the contribution-plugin refactor, were never registered in the committed lock file.

Retrigger CI (transient Docker Hub pull timeout on last run)

bc84f66

amirrza777 force-pushed the amir/csv-upload branch from 28eefe1 to bc84f66 Compare July 24, 2026 16:10

amirrza777 marked this pull request as ready for review July 24, 2026 16:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CSV import: upload support in the file explorer#24

CSV import: upload support in the file explorer#24
amirrza777 wants to merge 34 commits into
mde-optimiser:mainfrom
amirrza777:amir/csv-upload

amirrza777 commented Jul 14, 2026

Uh oh!

amirrza777 commented Jul 14, 2026

Uh oh!

szschaler commented Jul 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

amirrza777 commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

amirrza777 commented Jul 14, 2026

What this adds

Verified

Uh oh!

amirrza777 commented Jul 14, 2026

Uh oh!

szschaler commented Jul 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

amirrza777 commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants