Xopt External Connector¶

ParetoBench includes code to use its benchmark problems in Xopt and load data output from some of its multiobjective genetic algorithms. On this page, we will explore the use of the following ParetoBench features.

• XoptProblemWrapper: Use ParetoBench problems with Xopt.
• import_nsga2_history_dir: Load the output of one or more runs of NSGA2Generator into a paretobench.History.
• import_cnsga_history: Load the output of CNSGAGenerator into a paretobench.History.

In [1]:

import tempfile
import shutil
import numpy as np
import os
from xopt.base import Xopt
from xopt.evaluator import Evaluator
from xopt.generators.ga.cnsga import CNSGAGenerator
from xopt.generators.ga.nsga2 import NSGA2Generator

from paretobench.ext.xopt import (
    XoptProblemWrapper,
    import_cnsga_history,
    import_nsga2_history_dir,
)

import tempfile import shutil import numpy as np import os from xopt.base import Xopt from xopt.evaluator import Evaluator from xopt.generators.ga.cnsga import CNSGAGenerator from xopt.generators.ga.nsga2 import NSGA2Generator from paretobench.ext.xopt import ( XoptProblemWrapper, import_cnsga_history, import_nsga2_history_dir, )

In [2]:

# Make a directory for our optimization data (cleaned up at end of notebook)
dir = tempfile.mkdtemp()

# Make a directory for our optimization data (cleaned up at end of notebook) dir = tempfile.mkdtemp()

Using ParetoBench Problems in Xopt¶

In Xopt, problems are described by a VOCS object with the problems variables, objectives, and constraints as well as an evaluation function which maps dicts of variables to dicts of objectives / constraints. XoptProblemWrapper adapts any ParetoBench problem to this interface. The VOCS object is available from the wrapper's vocs property and the wrapper itself may be passed directly to Xopt's Evaluator.

In [3]:

# Wrap one of ParetoBench's problems for use in Xopt
prob = XoptProblemWrapper.from_line_fmt("ZDT3 (n=3)")
print(prob)

# The VOCS object is constructed from the problem's variable bounds, objectives, and constraints
prob.vocs

# Wrap one of ParetoBench's problems for use in Xopt prob = XoptProblemWrapper.from_line_fmt("ZDT3 (n=3)") print(prob) # The VOCS object is constructed from the problem's variable bounds, objectives, and constraints prob.vocs

XoptProblemWrapper(ZDT3 (n=3))

Out[3]:

VOCS(variables={'x0': ContinuousVariable(dtype=None, default_value=None, domain=[0.0, 1.0]), 'x1': ContinuousVariable(dtype=None, default_value=None, domain=[0.0, 1.0]), 'x2': ContinuousVariable(dtype=None, default_value=None, domain=[0.0, 1.0])}, objectives={'f0': MinimizeObjective(dtype=None), 'f1': MinimizeObjective(dtype=None)}, constraints={}, constants={}, observables={})

In [4]:

# The wrapper is a callable using Xopt's dict -> dict convention
print(prob({"x0": 1.0, "x1": 1.5, "x2": 2.0}))

# Arrays are also accepted for use with vectorized evaluators
print(prob({"x0": np.array([1.0, 0.5]), "x1": np.array([1.5, 0.75]), "x2": np.array([2.0, 1.0])}))

# The wrapper is a callable using Xopt's dict -> dict convention print(prob({"x0": 1.0, "x1": 1.5, "x2": 2.0})) # Arrays are also accepted for use with vectorized evaluators print(prob({"x0": np.array([1.0, 0.5]), "x1": np.array([1.5, 0.75]), "x2": np.array([2.0, 1.0])}))

{'f0': array([1.]), 'f1': array([12.65732361])}
{'f0': array([1. , 0.5]), 'f1': array([12.65732361,  6.76846256])}

Loading Data from NSGA2Generator¶

import_nsga2_history_dir may be used to load optimization data from the output_dir of NSGA2Generator into a paretobench.History object. We will now run a short optimization on the wrapped problem and load the data to demonstrate the use of the function.

In [5]:

# Generate optimization data for us to load later
population_size = 32
n_generations = 32
output_dir = os.path.join(dir, "nsga2_zdt3")

# Set up and run the optimizer while writing data to `output_dir`
xx = Xopt(
    generator=NSGA2Generator(vocs=prob.vocs, output_dir=output_dir, population_size=population_size),
    evaluator=Evaluator(function=prob, max_workers=population_size, vectorized=True),
    vocs=prob.vocs,
)
for _ in range(n_generations):
    xx.step()

# To avoid errors when running in notebook
xx.generator.close_log_file()

print(f"Ran NSGA2 for {n_generations} generations")

# Generate optimization data for us to load later population_size = 32 n_generations = 32 output_dir = os.path.join(dir, "nsga2_zdt3") # Set up and run the optimizer while writing data to `output_dir` xx = Xopt( generator=NSGA2Generator(vocs=prob.vocs, output_dir=output_dir, population_size=population_size), evaluator=Evaluator(function=prob, max_workers=population_size, vectorized=True), vocs=prob.vocs, ) for _ in range(n_generations): xx.step() # To avoid errors when running in notebook xx.generator.close_log_file() print(f"Ran NSGA2 for {n_generations} generations")

Ran NSGA2 for 32 generations

In [6]:

# Load the run into a History object
hist = import_nsga2_history_dir(output_dir, problem="ZDT3 (n=3)")

# Show some stats from it
print(hist)
print(f"Number of reports: {len(hist)}")
print(f"Individuals in final report: {len(hist.reports[-1])}")

# Load the run into a History object hist = import_nsga2_history_dir(output_dir, problem="ZDT3 (n=3)") # Show some stats from it print(hist) print(f"Number of reports: {len(hist)}") print(f"Individuals in final report: {len(hist.reports[-1])}")

History(problem='ZDT3 (n=3)', reports=32, vars=3, objs=[--], cons=[])
Number of reports: 32
Individuals in final report: 32

In [7]:

# The method also allows for easy plotting of optimization
# Here we show all individuals along with the true Pareto front
import_nsga2_history_dir(output_dir, problem="ZDT3 (n=3)").plot_obj_scatter(show_pf=True)

# The method also allows for easy plotting of optimization # Here we show all individuals along with the true Pareto front import_nsga2_history_dir(output_dir, problem="ZDT3 (n=3)").plot_obj_scatter(show_pf=True)

Out[7]:

(<Figure size 640x480 with 2 Axes>, <Axes: xlabel='f0', ylabel='f1'>)

Multiple runs of NSGA2Generator created by restarting from checkpoints can be combined into a single History object. Pass a list of the output directories (in the order they were run) or a glob pattern such as "my_optimization_*" to import_nsga2_history_dir. The VOCS in each directory must match and the generation numbering must be continuous across the runs (as produced by checkpoint restarts).

In [8]:

# Select the latest checkpoint file from the run
checkpoint_dir = os.path.join(output_dir, "checkpoints")
checkpoint_file = max(
    (os.path.join(checkpoint_dir, f) for f in os.listdir(checkpoint_dir)),
    key=os.path.getmtime,
)

# Run for more generations after checkpoint
xx = Xopt(
    generator=NSGA2Generator(vocs=prob.vocs, checkpoint_file=checkpoint_file),
    evaluator=Evaluator(function=prob, max_workers=population_size, vectorized=True),
    vocs=prob.vocs,
)
for _ in range(n_generations):
    xx.step()

# To avoid errors when running in notebook
xx.generator.close_log_file()

# Show the newly created file
print(f"Completed {n_generations} more generations")
print("Contents of project directory:")
for fname in sorted(os.listdir(dir)):
    print("  - " + fname)

# Select the latest checkpoint file from the run checkpoint_dir = os.path.join(output_dir, "checkpoints") checkpoint_file = max( (os.path.join(checkpoint_dir, f) for f in os.listdir(checkpoint_dir)), key=os.path.getmtime, ) # Run for more generations after checkpoint xx = Xopt( generator=NSGA2Generator(vocs=prob.vocs, checkpoint_file=checkpoint_file), evaluator=Evaluator(function=prob, max_workers=population_size, vectorized=True), vocs=prob.vocs, ) for _ in range(n_generations): xx.step() # To avoid errors when running in notebook xx.generator.close_log_file() # Show the newly created file print(f"Completed {n_generations} more generations") print("Contents of project directory:") for fname in sorted(os.listdir(dir)): print(" - " + fname)

Completed 32 more generations
Contents of project directory:
  - nsga2_zdt3
  - nsga2_zdt3_2

In [9]:

# Plot data from both runs (Note "*" character after output directory to match all outputs)
import_nsga2_history_dir(os.path.join(dir, "nsga2_zdt3*"), problem="ZDT3 (n=3)").plot_obj_scatter(show_pf=True)

# Plot data from both runs (Note "*" character after output directory to match all outputs) import_nsga2_history_dir(os.path.join(dir, "nsga2_zdt3*"), problem="ZDT3 (n=3)").plot_obj_scatter(show_pf=True)

Out[9]:

(<Figure size 640x480 with 2 Axes>, <Axes: xlabel='f0', ylabel='f1'>)

Loading Data from CNSGAGenerator¶

CNSGAGenerator saves each generation to a timestamped CSV file (cnsga_population_*.csv) when run with the output_path option. The function import_cnsga_history loads all of the population files in a directory into a History object. The population files do not contain the problem definition, so the VOCS object must be passed in as well. A path to a JSON file of the VOCS or the run's YAML config file (through the config keyword) also works.

In [10]:

# Generate optimization data using CNSGA
output_path = os.path.join(dir, "cnsga")
xx = Xopt(
    generator=CNSGAGenerator(vocs=prob.vocs, output_path=output_dir, population_size=population_size),
    evaluator=Evaluator(function=prob, max_workers=population_size, vectorized=True),
    vocs=prob.vocs,
)
for _ in range(n_generations):
    xx.step()

print(f"Ran CNSGA for {n_generations} generations")

# Generate optimization data using CNSGA output_path = os.path.join(dir, "cnsga") xx = Xopt( generator=CNSGAGenerator(vocs=prob.vocs, output_path=output_dir, population_size=population_size), evaluator=Evaluator(function=prob, max_workers=population_size, vectorized=True), vocs=prob.vocs, ) for _ in range(n_generations): xx.step() print(f"Ran CNSGA for {n_generations} generations")

Ran CNSGA for 32 generations

In [11]:

# Load the run into a History object
import_cnsga_history(output_dir, vocs=prob.vocs, problem="ZDT3 (n=3)").plot_obj_scatter(show_pf=True)

# Load the run into a History object import_cnsga_history(output_dir, vocs=prob.vocs, problem="ZDT3 (n=3)").plot_obj_scatter(show_pf=True)

Out[11]:

(<Figure size 640x480 with 2 Axes>, <Axes: xlabel='f0', ylabel='f1'>)

Cleanup¶

In [12]:

# Clean up optimization data
shutil.rmtree(dir)

# Clean up optimization data shutil.rmtree(dir)