Skip to content

Commit 67ce8d2

Browse files
committed
feat: modernize project infrastructure for cookiecutter data science compliance
- SIMPLIFY: Replace 3-service Docker setup with single IRIS container - MODERNIZE: Replace setup.py with pyproject.toml for modern Python packaging - AUTOMATE: Add comprehensive Makefile for development workflow - STANDARDIZE: Add .clinerules-data-scientist for project-specific best practices - OPTIMIZE: Implement uv support with pip fallback for faster dependency management - STREAMLINE: Update README with simplified quick start (make setup && make notebooks) This dramatically reduces complexity while preserving all 5 educational demos and IRIS IntegratedML integration value. Follows 'bare minimum' principle for easy developer adoption while maintaining data science best practices.
1 parent 7a4c48f commit 67ce8d2

File tree

8 files changed

+393
-378
lines changed

8 files changed

+393
-378
lines changed

.clinerules-data-scientist

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Data Science Project Rules for IntegratedML
2+
# Following cookiecutter data science best practices + modern tooling
3+
4+
## Dependency Management
5+
- Use `uv` for fast, reliable package management when possible
6+
- Pin exact versions in requirements.txt for reproducibility
7+
- Keep dependencies minimal - avoid bloat
8+
9+
## Code Organization
10+
- Heavy logic in .py modules under shared/ and demos/
11+
- Keep notebooks light - use for exploration and presentation
12+
- Use type hints and docstrings throughout
13+
14+
## Reproducibility
15+
- Log all preprocessing steps and data transformations
16+
- Save model artifacts with versioning information
17+
- Include confidence intervals in model benchmarks
18+
- Use DVC for large data/model versioning (when applicable)
19+
20+
## Development Workflow
21+
- Use Makefile for common tasks and automation
22+
- IRIS database via Docker, Python development locally
23+
- VS Code for notebook development (not Jupyter server)
24+
- Test-driven development with pytest
25+
26+
## Data Hygiene
27+
- Validate inputs at module boundaries
28+
- Document data assumptions and limitations
29+
- Keep raw data immutable
30+
- Track data lineage and transformations
31+
32+
## Model Development
33+
- Benchmark against baseline models
34+
- Report robust metrics with confidence intervals
35+
- Embed key plots and link to saved artifacts
36+
- Optimize for readability over cleverness
37+
38+
## Notebook Guidelines
39+
- Clear all outputs before committing
40+
- Use markdown for explanations and context
41+
- Import shared utilities rather than duplicating code
42+
- Include executive summary and next steps

Makefile

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# IntegratedML Flexible Model Integration - Development Automation
2+
# Simplified workflow for IRIS + local development
3+
4+
.PHONY: help setup start stop clean install test format lint notebooks
5+
6+
# Default target
7+
help:
8+
@echo "IntegratedML Flexible Model Integration - Available Commands:"
9+
@echo ""
10+
@echo "Setup & Environment:"
11+
@echo " setup - Complete project setup (dependencies + IRIS)"
12+
@echo " install - Install Python dependencies only"
13+
@echo " start - Start IRIS database"
14+
@echo " stop - Stop IRIS database"
15+
@echo " clean - Clean up containers and volumes"
16+
@echo ""
17+
@echo "Development:"
18+
@echo " notebooks - Open notebooks directory in VS Code"
19+
@echo " test - Run all tests"
20+
@echo " format - Format code with black"
21+
@echo " lint - Run linting checks"
22+
@echo ""
23+
@echo "Quick Start:"
24+
@echo " make setup && make notebooks"
25+
26+
# Environment setup
27+
setup: install start
28+
@echo "✅ Setup complete! Run 'make notebooks' to start exploring."
29+
30+
install:
31+
@echo "📦 Installing Python dependencies with uv..."
32+
@if command -v uv >/dev/null 2>&1; then \
33+
uv sync; \
34+
else \
35+
echo "⚠️ uv not found, falling back to pip..."; \
36+
pip install -r requirements.txt; \
37+
fi
38+
@echo "✅ Dependencies installed"
39+
40+
install-uv:
41+
@echo "🚀 Installing uv (ultra-fast Python package manager)..."
42+
curl -LsSf https://astral.sh/uv/install.sh | sh
43+
@echo "✅ uv installed! Restart your shell or run: source ~/.cargo/env"
44+
45+
# IRIS database management
46+
start:
47+
@echo "🚀 Starting IRIS database..."
48+
docker-compose up -d iris
49+
@echo "⏳ Waiting for IRIS to be ready..."
50+
@timeout 120 bash -c 'until docker-compose exec iris iris session iris -U USER "w \"Database ready\""; do sleep 2; done'
51+
@echo "✅ IRIS database is ready!"
52+
@echo " Management Portal: http://localhost:52773/csp/sys/UtilHome.csp"
53+
@echo " Database Port: localhost:1972"
54+
55+
stop:
56+
@echo "🛑 Stopping IRIS database..."
57+
docker-compose stop iris
58+
@echo "✅ IRIS stopped"
59+
60+
restart: stop start
61+
62+
# Development tools
63+
notebooks:
64+
@echo "📓 Opening notebooks in VS Code..."
65+
code notebooks/ demos/
66+
67+
test:
68+
@echo "🧪 Running tests..."
69+
@pytest demos/*/tests/ -v --tb=short
70+
@echo "✅ Tests completed"
71+
72+
format:
73+
@echo "🎨 Formatting code..."
74+
black .
75+
@echo "✅ Code formatted"
76+
77+
lint:
78+
@echo "🔍 Running linting..."
79+
flake8 . --max-line-length=88 --extend-ignore=E203,W503
80+
mypy shared/ --ignore-missing-imports
81+
@echo "✅ Linting completed"
82+
83+
# Cleanup
84+
clean:
85+
@echo "🧹 Cleaning up..."
86+
docker-compose down -v
87+
docker system prune -f
88+
@echo "✅ Cleanup completed"
89+
90+
# Demo-specific targets
91+
demo-credit:
92+
@echo "💳 Running Credit Risk demo..."
93+
python run_credit_risk_demo.py
94+
95+
demo-fraud:
96+
@echo "🔒 Running Fraud Detection demo..."
97+
python run_fraud_detection_demo.py
98+
99+
demo-sales:
100+
@echo "📈 Running Sales Forecasting demo..."
101+
python run_sales_forecasting_demo.py
102+
103+
demo-dna:
104+
@echo "🧬 Running DNA Similarity demo..."
105+
python run_dna_similarity_demo.py
106+
107+
# Quick demo runner
108+
demos: start
109+
@echo "🎬 Running all demos..."
110+
@$(MAKE) demo-credit
111+
@$(MAKE) demo-fraud
112+
@$(MAKE) demo-sales
113+
@$(MAKE) demo-dna
114+
@echo "✅ All demos completed!"
115+
116+
# Status check
117+
status:
118+
@echo "📊 System Status:"
119+
@echo ""
120+
@docker-compose ps
121+
@echo ""
122+
@if docker-compose exec iris iris session iris -U USER "w \"IRIS Status: Online\"" 2>/dev/null; then \
123+
echo "✅ IRIS: Online"; \
124+
else \
125+
echo "❌ IRIS: Offline"; \
126+
fi
127+
128+
# Development utilities
129+
dev-setup: install
130+
@echo "🛠️ Setting up development environment..."
131+
pip install pre-commit
132+
pre-commit install
133+
@echo "✅ Development tools configured"
134+
135+
# Container logs
136+
logs:
137+
docker-compose logs -f iris
138+
139+
# Database initialization
140+
init-db: start
141+
@echo "🗃️ Initializing database with sample data..."
142+
python setup_iris_integratedml.py
143+
@echo "✅ Database initialized"

README.md

Lines changed: 41 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -19,56 +19,68 @@ A demonstration framework for integrating custom machine learning models with In
1919
## Quick Start
2020

2121
### Prerequisites
22-
- Docker and Docker Compose
23-
- At least 8GB RAM for containers
24-
- 20GB free disk space
22+
- **Docker & Docker Compose** (for IRIS database)
23+
- **Python 3.8+** (for local development)
24+
- **VS Code** (recommended for notebooks)
25+
- At least 4GB RAM for IRIS container
2526

26-
### Docker Setup
27+
### 🚀 Simplified Setup
2728

2829
```bash
2930
# Clone the repository
3031
git clone https://github.com/intersystems-community/integratedml-flexible-model-integration.git
3132
cd integratedml-flexible-model-integration
3233

33-
# Initialize Docker environment
34-
chmod +x docker/docker-init.sh
35-
./docker/docker-init.sh
34+
# Complete setup (dependencies + IRIS database)
35+
make setup
3636

37-
# Configure environment
38-
cp .env.example .env
39-
# Edit .env with your preferred settings
40-
41-
# Start IRIS database and application services
42-
docker-compose up --build -d
37+
# Open notebooks in VS Code
38+
make notebooks
4339
```
4440

45-
### Run Demo Examples
41+
That's it! 🎉
42+
43+
### Manual Setup (Alternative)
4644

4745
```bash
48-
# Credit Risk Assessment with IRIS IntegratedML
49-
python run_credit_risk_demo.py
46+
# 1. Install Python dependencies
47+
pip install -r requirements.txt
5048

51-
# Fraud Detection with ensemble models
52-
python run_fraud_detection_demo.py
49+
# 2. Configure environment
50+
cp .env.example .env
5351

54-
# Sales Forecasting with hybrid models
55-
python run_sales_forecasting_demo.py
52+
# 3. Start IRIS database only
53+
docker-compose up -d iris
5654

57-
# DNA Similarity Analysis with sequence classification
58-
python run_dna_similarity_demo.py
55+
# 4. Open notebooks in VS Code
56+
code notebooks/ demos/
5957
```
6058

61-
### Alternative: Standalone Installation
59+
### Available Notebooks
6260

63-
```bash
64-
# For development without Docker
65-
pip install -r requirements.txt
66-
pip install -e .
61+
Explore these comprehensive demos directly in VS Code:
6762

68-
# Launch interactive notebooks
69-
jupyter notebook demos/credit_risk/notebooks/credit_risk_demo.ipynb
63+
- [**📓 Quick Start**](notebooks/Iris_IntegratedML_Quickstart.ipynb) - Get familiar with IRIS IntegratedML
64+
- [**💳 Credit Risk**](demos/credit_risk/notebooks/01_Credit_Risk_Complete_Demo.ipynb) - Financial risk assessment
65+
- [**🔒 Fraud Detection**](demos/fraud_detection/notebooks/01_Fraud_Detection_Complete_Demo.ipynb) - Real-time fraud prevention
66+
- [**📈 Sales Forecasting**](demos/sales_forecasting/notebooks/01_Sales_Forecasting_Complete_Demo.ipynb) - Revenue prediction
67+
- [**🧬 DNA Similarity**](demos/dna_similarity/notebooks/01_DNA_Similarity_Complete_Demo.ipynb) - Genomic analysis
68+
- [**📊 Time Series**](demos/time_series_native/notebooks/01_Time_Series_Native_Complete_Demo.ipynb) - Native IRIS time series
69+
70+
### 🛠️ Development Commands
71+
72+
```bash
73+
make help # Show all available commands
74+
make start # Start IRIS database
75+
make stop # Stop IRIS database
76+
make test # Run all tests
77+
make demos # Run all demo scripts
78+
make status # Check system status
7079
```
7180

81+
### What's New?
82+
**Simplified Development Workflow**: No more complex multi-container setup! Just IRIS database + local Python development in VS Code.
83+
7284
## Demo Examples
7385

7486
The framework includes four demonstration scenarios with varying complexity levels.

0 commit comments

Comments
 (0)