Skip to content

Format Handlers

Base Handler

base

Base classes for file format handlers.

This module defines the abstract interface that all file format handlers must implement, enabling a plug-and-play architecture for supporting different optimization result file formats.

Classes

BaseFormatHandler

Bases: ABC

Abstract base class for optimization result file format handlers.

Each format handler must implement methods to: - Detect if a file is in their format - Read optimization results from files - Write optimization results to files

Functions
can_handle abstractmethod classmethod 
can_handle(filepath: Path) -> bool

Check if this handler can read the given file.

Parameters:

Name Type Description Default
filepath Path

Path to file to check

 required

Returns:

Type Description
bool

True if this handler can read the file
Source code in optiscope/io/base.py 
35
36
37
38
39
40
41
42
43
44
45
46
47
@classmethod
@abstractmethod
def can_handle(cls, filepath: Path) -> bool:
    """
    Check if this handler can read the given file.

    Args:
        filepath: Path to file to check

    Returns:
        True if this handler can read the file
    """
    pass
read abstractmethod 
read(filepath: Path, **kwargs: Any) -> OptimizationResult

Read optimization results from file.

Parameters:

Name Type Description Default
filepath Path

Path to file to read

 required
**kwargs Any

Format-specific options

 {}

Returns:

Type Description
OptimizationResult

OptimizationResult object
Source code in optiscope/io/base.py 
49
50
51
52
53
54
55
56
57
58
59
60
61
@abstractmethod
def read(self, filepath: Path, **kwargs: Any) -> OptimizationResult:
    """
    Read optimization results from file.

    Args:
        filepath: Path to file to read
        **kwargs: Format-specific options

    Returns:
        OptimizationResult object
    """
    pass
write abstractmethod 
write(result: OptimizationResult, filepath: Path, **kwargs: Any) -> None

Write optimization results to file.

Parameters:

Name Type Description Default
result OptimizationResult

OptimizationResult to write

 required
filepath Path

Path to output file

 required
**kwargs Any

Format-specific options

 {}
Source code in optiscope/io/base.py 
63
64
65
66
67
68
69
70
71
72
73
@abstractmethod
def write(self, result: OptimizationResult, filepath: Path, **kwargs: Any) -> None:
    """
    Write optimization results to file.

    Args:
        result: OptimizationResult to write
        filepath: Path to output file
        **kwargs: Format-specific options
    """
    pass
validate_result 
validate_result(result: OptimizationResult) -> None

Validate that the result can be written by this handler.

Parameters:

Name Type Description Default
result OptimizationResult

OptimizationResult to validate

 required

Raises:

Type Description
ValueError

If result cannot be written
Source code in optiscope/io/base.py 
75
76
77
78
79
80
81
82
83
84
85
86
def validate_result(self, result: OptimizationResult) -> None:
    """
    Validate that the result can be written by this handler.

    Args:
        result: OptimizationResult to validate

    Raises:
        ValueError: If result cannot be written
    """
    if result.n_points == 0:
        raise ValueError("Cannot write empty result")

StructuredFormatHandler

Bases: BaseFormatHandler

Base class for handlers of structured formats with explicit metadata.

Structured formats (like JSON, YAML, HDF5) can store metadata alongside data, making round-trip reading/writing lossless.

Functions
read_metadata abstractmethod 
read_metadata(filepath: Path) -> dict[str, Any]

Read just the metadata from file without loading all data.

Parameters:

Name Type Description Default
filepath Path

Path to file

 required

Returns:

Type Description
dict[str, Any]

Dictionary of metadata
Source code in optiscope/io/base.py 
124
125
126
127
128
129
130
131
132
133
134
135
@abstractmethod
def read_metadata(self, filepath: Path) -> dict[str, Any]:
    """
    Read just the metadata from file without loading all data.

    Args:
        filepath: Path to file

    Returns:
        Dictionary of metadata
    """
    pass

TabularFormatHandler

Bases: BaseFormatHandler

Base class for handlers of tabular formats (CSV, Excel, etc.).

Tabular formats typically store only data without metadata, requiring inference or separate metadata files.

Functions
infer_column_categories 
infer_column_categories(columns: list[str], design_var_prefix: str | None = None, objective_prefix: str | None = None, equality_constraint_prefix: str | None = None, inequality_constraint_prefix: str | None = None, observable_prefix: str | None = None) -> tuple[dict[str, list[str]], dict[str, str]]

Infer which columns belong to which category based on naming.

Parameters:

Name Type Description Default
columns list[str]

List of column names

 required
design_var_prefix str | None

Prefix for design variables

 None
objective_prefix str | None

Prefix for objectives

 None
equality_constraint_prefix str | None

Prefix for constraints

 None
inequality_constraint_prefix str | None

Prefix for constraints

 None
observable_prefix str | None

Prefix for observables

 None

Returns:

Type Description
dict[str, list[str]]

A tuple containing:
dict[str, str]
  • Dictionary mapping categories to stripped column names
tuple[dict[str, list[str]], dict[str, str]]
  • Dictionary mapping original column names to stripped column names
Source code in optiscope/io/base.py 
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
def infer_column_categories(
    self,
    columns: list[str],
    design_var_prefix: str | None = None,
    objective_prefix: str | None = None,
    equality_constraint_prefix: str | None = None,
    inequality_constraint_prefix: str | None = None,
    observable_prefix: str | None = None,
) -> tuple[dict[str, list[str]], dict[str, str]]:
    """
    Infer which columns belong to which category based on naming.

    Args:
        columns: List of column names
        design_var_prefix: Prefix for design variables
        objective_prefix: Prefix for objectives
        equality_constraint_prefix: Prefix for constraints
        inequality_constraint_prefix: Prefix for constraints
        observable_prefix: Prefix for observables

    Returns:
        A tuple containing:
        - Dictionary mapping categories to stripped column names
        - Dictionary mapping original column names to stripped column names
    """
    design_var_prefix = design_var_prefix or self.default_design_var_prefix
    objective_prefix = objective_prefix or self.default_objective_prefix
    equality_constraint_prefix = (
        equality_constraint_prefix or self.default_equality_constraint_prefix
    )
    inequality_constraint_prefix = (
        inequality_constraint_prefix or self.default_inequality_constraint_prefix
    )
    observable_prefix = observable_prefix or self.default_observable_prefix

    categorized = {
        "design_variables": [],
        "objectives": [],
        "equality_constraints": [],
        "inequality_constraints": [],
        "observables": [],
        "unknown": [],
    }
    rename_map = {}

    for col in columns:
        col_lower = col.lower()
        if col_lower.startswith(design_var_prefix.lower()):
            new_name = col[len(design_var_prefix) :]
            categorized["design_variables"].append(new_name)
            rename_map[col] = new_name
        elif col_lower.startswith(objective_prefix.lower()):
            new_name = col[len(objective_prefix) :]
            categorized["objectives"].append(new_name)
            rename_map[col] = new_name
        elif col_lower.startswith(equality_constraint_prefix.lower()):
            new_name = col[len(equality_constraint_prefix) :]
            categorized["equality_constraints"].append(new_name)
            rename_map[col] = new_name
        elif col_lower.startswith(inequality_constraint_prefix.lower()):
            new_name = col[len(inequality_constraint_prefix) :]
            categorized["inequality_constraints"].append(new_name)
            rename_map[col] = new_name
        elif col_lower.startswith(observable_prefix.lower()):
            new_name = col[len(observable_prefix) :]
            categorized["observables"].append(new_name)
            rename_map[col] = new_name
        else:
            categorized["unknown"].append(col)
            rename_map[col] = col

    return categorized, rename_map
create_default_metadata 
create_default_metadata(n_points: int, categorized_columns: dict[str, list[str]]) -> ProblemMetadata

Create default problem metadata when none is available.

Parameters:

Name Type Description Default
n_points int

Number of data points

 required
categorized_columns dict[str, list[str]]

Categorized column names

 required

Returns:

Type Description
ProblemMetadata

ProblemMetadata with inferred values
Source code in optiscope/io/base.py 
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
def create_default_metadata(
    self, n_points: int, categorized_columns: dict[str, list[str]]
) -> ProblemMetadata:
    """
    Create default problem metadata when none is available.

    Args:
        n_points: Number of data points
        categorized_columns: Categorized column names

    Returns:
        ProblemMetadata with inferred values
    """
    return ProblemMetadata(
        name="Imported Data",
        description="Data imported from a tabular file.",
        solver="Unknown",
        solver_version=None,
        n_design_variables=len(categorized_columns.get("design_variables", [])),
        n_objectives=len(categorized_columns.get("objectives", [])),
        n_equality_constraints=len(categorized_columns.get("equality_constraints", [])),
        n_inequality_constraints=len(categorized_columns.get("inequality_constraints", [])),
        computation_time=None,
        n_evaluations=None,
        metadata={
            "n_points": n_points,
            "inferred": True,
            "unknown_columns": categorized_columns.get("unknown", []),
        },
    )

FormatDetectionError

Bases: Exception

Exception raised when file format cannot be detected.

FormatReadError

Bases: Exception

Exception raised when file cannot be read.

FormatWriteError

Bases: Exception

Exception raised when file cannot be written.

CSV Handler

csv_handler

CSV file format handler for optimization results.

Supports reading and writing optimization results in CSV format with configurable column naming conventions and optional metadata sidecar files.

Classes

CSVHandler

Bases: TabularFormatHandler

Handler for CSV files containing optimization results.

CSV files can be read with automatic column categorization based on naming conventions. Metadata can be stored in a separate JSON sidecar file.

Functions
can_handle classmethod 
can_handle(filepath: Path) -> bool

Check if file is a CSV.

Source code in optiscope/io/csv_handler.py 
39
40
41
42
43
44
45
46
47
48
49
50
@classmethod
def can_handle(cls, filepath: Path) -> bool:
    """Check if file is a CSV."""
    if filepath.suffix.lower() != ".csv":
        return False

    try:
        # Try to read first few lines
        pd.read_csv(filepath, nrows=5)
        return True
    except Exception:
        return False
read 
read(filepath: Path, design_var_prefix: str | None = None, objective_prefix: str | None = None, inequality_constraint_prefix: str | None = None, equality_constraint_prefix: str | None = None, observable_prefix: str | None = None, metadata_file: Path | None = None, **kwargs: Any) -> OptimizationResult

Read optimization results from CSV file.

Parameters:

Name Type Description Default
filepath Path

Path to CSV file

 required
design_var_prefix str | None

Prefix for design variable columns

 None
objective_prefix str | None

Prefix for objective columns

 None
inequality_constraint_prefix str | None

Prefix for inequality constraint columns (g(x) <= 0)

 None
equality_constraint_prefix str | None

Prefix for equality constraint columns (h(x) = 0)

 None
observable_prefix str | None

Prefix for observable columns

 None
metadata_file Path | None

Optional path to metadata JSON file

 None
**kwargs Any

Additional arguments passed to pd.read_csv

 {}

Returns:

Type Description
OptimizationResult

OptimizationResult object
Source code in optiscope/io/csv_handler.py 
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
def read(
    self,
    filepath: Path,
    design_var_prefix: str | None = None,
    objective_prefix: str | None = None,
    inequality_constraint_prefix: str | None = None,
    equality_constraint_prefix: str | None = None,
    observable_prefix: str | None = None,
    metadata_file: Path | None = None,
    **kwargs: Any,
) -> OptimizationResult:
    """
    Read optimization results from CSV file.

    Args:
        filepath: Path to CSV file
        design_var_prefix: Prefix for design variable columns
        objective_prefix: Prefix for objective columns
        inequality_constraint_prefix: Prefix for inequality constraint columns (g(x) <= 0)
        equality_constraint_prefix: Prefix for equality constraint columns (h(x) = 0)
        observable_prefix: Prefix for observable columns
        metadata_file: Optional path to metadata JSON file
        **kwargs: Additional arguments passed to pd.read_csv

    Returns:
        OptimizationResult object
    """
    try:
        # Read CSV
        df = pd.read_csv(filepath, **kwargs)

        if df.empty:
            raise FormatReadError("CSV file is empty")

        # Categorize columns
        categorized, rename_map = self.infer_column_categories(
            df.columns.tolist(),
            design_var_prefix,
            objective_prefix,
            inequality_constraint_prefix,
            equality_constraint_prefix,
            observable_prefix,
        )

        # Rename columns to strip prefixes
        df.rename(columns=rename_map, inplace=True)

        # Extract data by category
        design_vars = (
            df[categorized["design_variables"]]
            if categorized["design_variables"]
            else pd.DataFrame()
        )
        objectives = (
            df[categorized["objectives"]] if categorized["objectives"] else pd.DataFrame()
        )
        inequality_constraints = (
            df[categorized["inequality_constraints"]]
            if categorized["inequality_constraints"]
            else pd.DataFrame()
        )
        equality_constraints = (
            df[categorized["equality_constraints"]]
            if categorized["equality_constraints"]
            else pd.DataFrame()
        )
        observables = (
            df[categorized["observables"]] if categorized["observables"] else pd.DataFrame()
        )

        # Handle unknown columns as observables
        unknown_df = df[categorized["unknown"]] if categorized["unknown"] else pd.DataFrame()
        if not unknown_df.empty:
            observables = pd.concat([observables, unknown_df], axis=1)

        # Load or create metadata
        if metadata_file and metadata_file.exists():
            problem_metadata = self._load_metadata_file(metadata_file)
        else:
            # Check for sidecar metadata file
            sidecar_path = Path(filepath).with_suffix(".meta.json")
            if sidecar_path.exists():
                problem_metadata = self._load_metadata_file(sidecar_path)
            else:
                problem_metadata = self.create_default_metadata(len(df), categorized)

        # Create result
        result = OptimizationResult(
            problem_metadata=problem_metadata,
            design_variables=design_vars,
            objectives=objectives,
            inequality_constraints=inequality_constraints,
            equality_constraints=equality_constraints,
            observables=observables,
        )

        # Add basic metadata for variables
        self._add_inferred_metadata(result, categorized)

        return result

    except pd.errors.EmptyDataError:
        raise FormatReadError("CSV file is empty or malformed")
    except Exception as e:
        raise FormatReadError(f"Failed to read CSV: {str(e)}") from e
write 
write(result: OptimizationResult, filepath: Path, include_metadata: bool = True, **kwargs: Any) -> None

Write optimization results to CSV file.

Parameters:

Name Type Description Default
result OptimizationResult

OptimizationResult to write

 required
filepath Path

Output CSV path

 required
include_metadata bool

Whether to write metadata sidecar file

 True
**kwargs Any

Additional arguments passed to pd.to_csv

 {}
Source code in optiscope/io/csv_handler.py 
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
def write(
    self,
    result: OptimizationResult,
    filepath: Path,
    include_metadata: bool = True,
    **kwargs: Any,
) -> None:
    """
    Write optimization results to CSV file.

    Args:
        result: OptimizationResult to write
        filepath: Output CSV path
        include_metadata: Whether to write metadata sidecar file
        **kwargs: Additional arguments passed to pd.to_csv
    """
    try:
        # Combine all data
        data_frames = []

        if not result.design_variables.empty:
            data_frames.append(result.design_variables)
        if not result.objectives.empty:
            data_frames.append(result.objectives)
        if not result.inequality_constraints.empty:
            data_frames.append(result.inequality_constraints)
        if not result.equality_constraints.empty:
            data_frames.append(result.equality_constraints)
        if not result.observables.empty:
            data_frames.append(result.observables)

        if not data_frames:
            raise FormatWriteError("No data to write")

        df = pd.concat(data_frames, axis=1)

        # Write CSV
        df.to_csv(filepath, index=False, **kwargs)

        # Write metadata sidecar if requested
        if include_metadata:
            metadata_path = Path(filepath).with_suffix(".meta.json")
            self._write_metadata_file(result, metadata_path)

    except Exception as e:
        raise FormatWriteError(f"Failed to write CSV: {str(e)}") from e

JSON Handler

json_handler

JSON file format handler for optimization results.

Supports reading and writing optimization results in JSON format with full metadata preservation and support for result sets.

Classes

JSONHandler

Bases: StructuredFormatHandler

Handler for JSON files containing optimization results.

JSON format preserves all metadata, variable information, and result sets. This is the recommended format for complete data preservation.

Functions
can_handle classmethod 
can_handle(filepath: Path) -> bool

Check if file is a valid JSON optimization result.

Source code in optiscope/io/json_handler.py 
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
@classmethod
def can_handle(cls, filepath: Path) -> bool:
    """Check if file is a valid JSON optimization result."""
    if filepath.suffix.lower() != ".json":
        return False

    try:
        with open(filepath) as f:
            data = json.load(f)

        # Check for required structure
        return isinstance(data, dict) and (
            "problem_metadata" in data
            or "design_variables" in data
            or "objectives" in data
            or "inequality_constraints" in data
            or "equality_constraints" in data
        )
    except Exception:
        return False
read 
read(filepath: Path, **kwargs: Any) -> OptimizationResult

Read optimization results from JSON file.

Parameters:

Name Type Description Default
filepath Path

Path to JSON file

 required
**kwargs Any

Unused (for interface compatibility)

 {}

Returns:

Type Description
OptimizationResult

OptimizationResult object
Source code in optiscope/io/json_handler.py 
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
def read(self, filepath: Path, **kwargs: Any) -> OptimizationResult:
    """
    Read optimization results from JSON file.

    Args:
        filepath: Path to JSON file
        **kwargs: Unused (for interface compatibility)

    Returns:
        OptimizationResult object
    """
    try:
        with open(filepath) as f:
            data = json.load(f)

        # Parse problem metadata
        problem_metadata = None
        if "problem_metadata" in data:
            meta_dict = data["problem_metadata"]
            # Convert ISO datetime strings
            if "run_date" in meta_dict and isinstance(meta_dict["run_date"], str):
                meta_dict["run_date"] = datetime.fromisoformat(meta_dict["run_date"])
            problem_metadata = ProblemMetadata(**meta_dict)

        # Parse data sections
        design_vars = self._parse_dataframe(data.get("design_variables", {}))
        objectives = self._parse_dataframe(data.get("objectives", {}))
        inequality_constraints = self._parse_dataframe(data.get("inequality_constraints", {}))
        equality_constraints = self._parse_dataframe(data.get("equality_constraints", {}))
        observables = self._parse_dataframe(data.get("observables", {}))

        # Parse custom data
        custom_data = {}
        if "custom_data" in data:
            for name, df_data in data["custom_data"].items():
                custom_data[name] = self._parse_dataframe(df_data)

        # Create result
        result = OptimizationResult(
            problem_metadata=problem_metadata,
            design_variables=design_vars,
            objectives=objectives,
            inequality_constraints=inequality_constraints,
            equality_constraints=equality_constraints,
            observables=observables,
            custom_data=custom_data,
        )

        # Parse variable metadata
        if "variable_metadata" in data:
            for var_name, meta_dict in data["variable_metadata"].items():
                meta = self._parse_variable_metadata(meta_dict)
                result.add_variable_metadata(meta)

        # Parse result sets
        if "sets" in data:
            for set_name, set_dict in data["sets"].items():
                result_set = ResultSet.from_dict(set_dict)
                result._set_manager.add_set(result_set)

        return result

    except json.JSONDecodeError as e:
        raise FormatReadError(f"Invalid JSON: {str(e)}") from e
    except Exception as e:
        raise FormatReadError(f"Failed to read JSON: {str(e)}") from e
write 
write(result: OptimizationResult, filepath: Path, indent: int = 2, **kwargs: Any) -> None

Write optimization results to JSON file.

Parameters:

Name Type Description Default
result OptimizationResult

OptimizationResult to write

 required
filepath Path

Output JSON path

 required
indent int

JSON indentation level

 2
**kwargs Any

Unused (for interface compatibility)

 {}
Source code in optiscope/io/json_handler.py 
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
def write(
    self, result: OptimizationResult, filepath: Path, indent: int = 2, **kwargs: Any
) -> None:
    """
    Write optimization results to JSON file.

    Args:
        result: OptimizationResult to write
        filepath: Output JSON path
        indent: JSON indentation level
        **kwargs: Unused (for interface compatibility)
    """
    try:
        data = {
            "format_version": "1.0",
            "problem_metadata": result.problem_metadata.model_dump(mode="json"),
        }

        # Add data sections
        if not result.design_variables.empty:
            data["design_variables"] = self._dataframe_to_dict(result.design_variables)

        if not result.objectives.empty:
            data["objectives"] = self._dataframe_to_dict(result.objectives)

        if not result.inequality_constraints.empty:
            data["inequality_constraints"] = self._dataframe_to_dict(
                result.inequality_constraints
            )

        if not result.equality_constraints.empty:
            data["equality_constraints"] = self._dataframe_to_dict(result.equality_constraints)

        if not result.observables.empty:
            data["observables"] = self._dataframe_to_dict(result.observables)

        # Add custom data
        if result._custom_data:
            data["custom_data"] = {
                name: self._dataframe_to_dict(df) for name, df in result._custom_data.items()
            }

        # Add variable metadata
        if result._variable_metadata:
            data["variable_metadata"] = {
                name: meta.model_dump(mode="json")
                for name, meta in result._variable_metadata.items()
            }

        # Add result sets
        if result.sets:
            data["sets"] = {name: s.to_dict() for name, s in result.sets.items()}

        # Write JSON
        with open(filepath, "w") as f:
            json.dump(data, f, indent=indent, default=str)

    except Exception as e:
        raise FormatWriteError(f"Failed to write JSON: {str(e)}") from e
read_metadata 
read_metadata(filepath: Path) -> dict[str, Any]

Read just metadata from JSON file.

Parameters:

Name Type Description Default
filepath Path

Path to JSON file

 required

Returns:

Type Description
dict[str, Any]

Dictionary containing metadata
Source code in optiscope/io/json_handler.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
def read_metadata(self, filepath: Path) -> dict[str, Any]:
    """
    Read just metadata from JSON file.

    Args:
        filepath: Path to JSON file

    Returns:
        Dictionary containing metadata
    """
    with open(filepath) as f:
        data = json.load(f)

    metadata = {
        "problem_metadata": data.get("problem_metadata", {}),
        "n_points": 0,
        "variables": list(data.get("variable_metadata", {}).keys()),
        "sets": list(data.get("sets", {}).keys()),
    }

    # Count points from any available data
    for key in [
        "design_variables",
        "objectives",
        "inequality_constraints",
        "equality_constraints",
        "observables",
    ]:
        if key in data and "data" in data[key]:
            metadata["n_points"] = len(data[key]["data"])
            break

    return metadata

MIDACO Handler

midaco_handler

MIDACO file format handler.

MIDACO is a solver for mixed-integer optimization. It outputs results in two file formats: - SOLUTION.TXT: Complete history of solutions evaluated - PARETO.TXT: Pareto front approximation (for multi-objective problems)

File Format: - Header lines start with # - First data line: O (objectives), M (constraints), N (design vars), [PSIZE] - Following lines: F(1:O) G(1:M) X(1:N) values space-separated

Classes

MIDACoHandler

Bases: TabularFormatHandler

Handler for MIDACO solution files.

Supports both SOLUTION.TXT (history) and PARETO.TXT (Pareto front) formats.

Functions
can_handle classmethod 
can_handle(filepath: Path) -> bool

Check if file is a MIDACO format.

Source code in optiscope/io/midaco_handler.py 
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
@classmethod
def can_handle(cls, filepath: Path) -> bool:
    """Check if file is a MIDACO format."""
    if filepath.suffix.lower() != ".txt":
        return False

    # Check for MIDACO-specific markers
    try:
        with open(filepath) as f:
            content = f.read(500)  # Read first 500 chars

            # Look for MIDACO-specific header patterns
            if any(
                marker in content
                for marker in [
                    "SOLUTION HISTORY",
                    "pareto front approximation",
                    "MIDACO solution",
                ]
            ):
                return True

            # Check for O M N pattern
            lines = content.split("\n")
            for line in lines:
                if line.strip() and not line.strip().startswith("#"):
                    # Should be O M N [PSIZE] line
                    parts = line.split()
                    if len(parts) >= 3:
                        try:
                            # Try to parse as integers
                            int(parts[0]), int(parts[1]), int(parts[2])
                            return True
                        except ValueError:
                            pass
                    break
    except Exception:
        return False

    return False
read 
read(filepath: Path, parse_pareto: bool = True, **kwargs: Any) -> OptimizationResult

Read MIDACO solution file.

Parameters:

Name Type Description Default
filepath Path

Path to MIDACO file (SOLUTION.TXT or PARETO.TXT)

 required
parse_pareto bool

If True and this is PARETO.TXT, automatically create 'pareto' and 'midaco_best' result sets

 True
**kwargs Any

Additional arguments (unused)

 {}

Returns:

Type Description
OptimizationResult

OptimizationResult object
Source code in optiscope/io/midaco_handler.py 
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
def read(self, filepath: Path, parse_pareto: bool = True, **kwargs: Any) -> OptimizationResult:
    """
    Read MIDACO solution file.

    Args:
        filepath: Path to MIDACO file (SOLUTION.TXT or PARETO.TXT)
        parse_pareto: If True and this is PARETO.TXT, automatically create
                     'pareto' and 'midaco_best' result sets
        **kwargs: Additional arguments (unused)

    Returns:
        OptimizationResult object
    """
    try:
        with open(filepath) as f:
            lines = f.readlines()

        # Parse header to determine file type and dimensions
        file_type, n_obj, n_const, n_vars, psize = self._parse_header(lines)

        # Extract data lines (non-comment, non-empty)
        data_lines = []
        for line in lines:
            stripped = line.strip()
            if stripped and not stripped.startswith("#"):
                # Skip dimension line (O M N [PSIZE])
                parts = stripped.split()
                if len(parts) == n_obj + n_const + n_vars:
                    try:
                        values = [float(x) for x in parts]
                        data_lines.append(values)
                    except ValueError:
                        continue

        if not data_lines:
            raise FormatReadError("No valid data lines found in MIDACO file")

        # Convert to numpy array
        data = np.array(data_lines)
        n_points = len(data)

        # Split into objectives, constraints, design variables
        idx = 0

        # Objectives F(1:O)
        objectives_data = data[:, idx : idx + n_obj]
        idx += n_obj

        # Constraints G(1:M)
        if n_const > 0:
            constraints_data = data[:, idx : idx + n_const]
            idx += n_const
        else:
            constraints_data = None

        # Design variables X(1:N)
        design_vars_data = data[:, idx : idx + n_vars]

        # Create DataFrames with proper column names
        obj_names = [f"f{i + 1}" for i in range(n_obj)]
        var_names = [f"x{i + 1}" for i in range(n_vars)]

        objectives_df = pd.DataFrame(objectives_data, columns=obj_names)
        design_vars_df = pd.DataFrame(design_vars_data, columns=var_names)

        # MIDACO constraints are inequality (g(x) <= 0)
        if constraints_data is not None:
            const_names = [f"g{i + 1}" for i in range(n_const)]
            constraints_df = pd.DataFrame(constraints_data, columns=const_names)
        else:
            constraints_df = pd.DataFrame()

        # Create problem metadata
        problem_metadata = ProblemMetadata(
            name=f"MIDACO {file_type}",
            description=f"Loaded from {filepath.name}",
            solver="MIDACO",
            n_design_variables=n_vars,
            n_objectives=n_obj,
            n_inequality_constraints=n_const,
            n_equality_constraints=0,
            n_evaluations=n_points,
            metadata={"file_type": file_type, "source_file": filepath.name, "psize": psize},
        )  # type: ignore

        # Create result
        result = OptimizationResult(
            problem_metadata=problem_metadata,
            design_variables=design_vars_df,
            objectives=objectives_df,
            inequality_constraints=constraints_df,
            equality_constraints=pd.DataFrame(),
        )

        # Add variable metadata
        for i, name in enumerate(obj_names):
            result.add_variable_metadata(
                Objective(name=name, description=f"Objective {i + 1} from MIDACO")  # type: ignore
            )

        for i, name in enumerate(var_names):
            result.add_variable_metadata(
                DesignVariable(name=name, description=f"Design variable {i + 1} from MIDACO")  # type: ignore
            )

        for i, name in enumerate(const_names) if constraints_data is not None else []:
            result.add_variable_metadata(
                Constraint(name=name, description=f"Inequality constraint {i + 1} (g <= 0)")  # type: ignore
            )

        # For PARETO.TXT files, create result sets
        if parse_pareto and file_type == "PARETO":
            # First solution is the MIDACO solution (best found)
            result.create_set(
                name="midaco_best",
                indices=[0],
                created_by="midaco_import",
                description="Best solution found by MIDACO",
                color="#FF6B6B",
            )

            # All other solutions are the Pareto front approximation
            if n_points > 1:
                result.create_set(
                    name="pareto_front",
                    indices=list(range(1, n_points)),
                    created_by="midaco_import",
                    description="Pareto front approximation from MIDACO",
                    color="#4ECDC4",
                )

        return result

    except Exception as e:
        raise FormatReadError(f"Failed to read MIDACO file: {str(e)}") from e
write 
write(result: OptimizationResult, filepath: Path, file_type: str = 'SOLUTION', include_pareto_set: str | None = None, **kwargs: Any) -> None

Write optimization results to MIDACO format.

Parameters:

Name Type Description Default
result OptimizationResult

OptimizationResult to write

 required
filepath Path

Output file path

 required
file_type str

"SOLUTION" for history or "PARETO" for Pareto front

 'SOLUTION'
include_pareto_set str | None

Name of result set to write as Pareto front (only used if file_type="PARETO")

 None
**kwargs Any

Additional arguments (unused)

 {}
Source code in optiscope/io/midaco_handler.py 
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
def write(
    self,
    result: OptimizationResult,
    filepath: Path,
    file_type: str = "SOLUTION",
    include_pareto_set: str | None = None,
    **kwargs: Any,
) -> None:
    """
    Write optimization results to MIDACO format.

    Args:
        result: OptimizationResult to write
        filepath: Output file path
        file_type: "SOLUTION" for history or "PARETO" for Pareto front
        include_pareto_set: Name of result set to write as Pareto front
                           (only used if file_type="PARETO")
        **kwargs: Additional arguments (unused)
    """
    try:
        n_obj = len(result.objectives.columns)
        n_const = len(result.inequality_constraints.columns)
        n_vars = len(result.design_variables.columns)
        n_points = result.n_points

        with open(filepath, "w") as f:
            # Write header
            if file_type == "SOLUTION":
                f.write("###################################################\n")
                f.write("### This file contains the history of solutions ###\n")
                f.write("###################################################\n")
                f.write("### SOLUTION FORMAT:  F(1:O)  G(1:M)  X(1:N)    ###\n")
                f.write("###################################################\n")
            else:  # PARETO
                f.write("#########################################################\n")
                f.write("### This file contains the pareto front approximation ###\n")
                f.write("#########################################################\n")
                f.write("### Solution format:     F(1:O)    G(1:M)    X(1:N)   ###\n")
                f.write("#########################################################\n")

            f.write("#                              \n")
            if file_type == "SOLUTION":
                f.write("#        O         M         N \n")
            else:
                f.write("#        O         M         N     PSIZE\n")
            f.write("#                              \n")

            # Write dimensions
            if file_type == "SOLUTION":
                f.write(f"{n_obj:10d}{n_const:10d}{n_vars:10d}\n")
            else:
                psize = (
                    n_points
                    if include_pareto_set is None
                    else len(result.get_set(include_pareto_set).indices)
                )
                f.write(f"{n_obj:10d}{n_const:10d}{n_vars:10d}{psize:10d}\n")

            f.write("#                       \n")

            # Write data
            if file_type == "SOLUTION":
                f.write("#        SOLUTION HISTORY (in chronological order)\n")
                f.write("#                       \n")
                indices = range(n_points)
            else:  # PARETO
                if include_pareto_set and include_pareto_set in result.sets:
                    f.write("#                       \n")
                    f.write("#        MIDACO solution\n")
                    f.write("#                       \n")

                    # Write best solution (first point of set)
                    pareto_set = result.get_set(include_pareto_set)
                    best_idx = pareto_set.indices[0]
                    self._write_solution_line(f, result, best_idx)

                    f.write("#                       \n")
                    f.write("#        All non-dominated solutions found by MIDACO\n")
                    f.write("#                       \n")

                    # Write rest of Pareto front
                    indices = pareto_set.indices[1:] if len(pareto_set.indices) > 1 else []
                else:
                    f.write("#        All solutions\n")
                    f.write("#                       \n")
                    indices = range(n_points)

            # Write solution lines
            for idx in indices:
                self._write_solution_line(f, result, idx)

    except Exception as e:
        raise FormatWriteError(f"Failed to write MIDACO file: {str(e)}") from e

Functions

read_midaco_pair 

read_midaco_pair(solution_file: Path, pareto_file: Path) -> tuple[OptimizationResult, OptimizationResult]

Read both SOLUTION.TXT and PARETO.TXT files from MIDACO.

Convenience function to load both files at once.

Parameters:

Name Type Description Default
solution_file Path

Path to SOLUTION.TXT

 required
pareto_file Path

Path to PARETO.TXT

 required

Returns:

Type Description
tuple[OptimizationResult, OptimizationResult]

Tuple of (solution_history, pareto_front) OptimizationResult objects
Source code in optiscope/io/midaco_handler.py 
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
def read_midaco_pair(
    solution_file: Path, pareto_file: Path
) -> tuple[OptimizationResult, OptimizationResult]:
    """
    Read both SOLUTION.TXT and PARETO.TXT files from MIDACO.

    Convenience function to load both files at once.

    Args:
        solution_file: Path to SOLUTION.TXT
        pareto_file: Path to PARETO.TXT

    Returns:
        Tuple of (solution_history, pareto_front) OptimizationResult objects
    """
    handler = MIDACoHandler()

    solution_result = handler.read(solution_file, parse_pareto=False)
    pareto_result = handler.read(pareto_file, parse_pareto=True)

    return solution_result, pareto_result

detect_midaco_files 

detect_midaco_files(directory: Path) -> dict[str, Path]

Detect MIDACO files in a directory.

Parameters:

Name Type Description Default
directory Path

Directory to search

 required

Returns:

Type Description
dict[str, Path]

Dictionary mapping file types to paths:
dict[str, Path]

{'SOLUTION': path, 'PARETO': path}
Source code in optiscope/io/midaco_handler.py 
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
def detect_midaco_files(directory: Path) -> dict[str, Path]:
    """
    Detect MIDACO files in a directory.

    Args:
        directory: Directory to search

    Returns:
        Dictionary mapping file types to paths:
        {'SOLUTION': path, 'PARETO': path}
    """
    directory = Path(directory)
    found_files = {}

    # Common MIDACO filenames
    solution_names = ["SOLUTION.TXT", "solution.txt", "SOLUTION.OUT", "solution.out"]
    pareto_names = ["PARETO.TXT", "pareto.txt", "PARETO.OUT", "pareto.out"]

    for name in solution_names:
        filepath = directory / name
        if filepath.exists():
            found_files["SOLUTION"] = filepath
            break

    for name in pareto_names:
        filepath = directory / name
        if filepath.exists():
            found_files["PARETO"] = filepath
            break

    # If not found by common names, search all .txt files
    if "SOLUTION" not in found_files or "PARETO" not in found_files:
        handler = MIDACoHandler()
        for txt_file in directory.glob("*.txt"):
            if handler.can_handle(txt_file):
                with open(txt_file) as f:
                    content = f.read(500)
                    if "pareto front approximation" in content.lower():
                        found_files["PARETO"] = txt_file
                    elif "SOLUTION HISTORY" in content:
                        found_files["SOLUTION"] = txt_file

    return found_files