Skip to content

Commit

Permalink
Update algorithm.md - Format
Browse files Browse the repository at this point in the history
  • Loading branch information
Aero-Spec authored Sep 17, 2024
1 parent 2588ae1 commit edc34ae
Showing 1 changed file with 29 additions and 43 deletions.
72 changes: 29 additions & 43 deletions docs/src/algorithm.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ The DIRECT algorithm incrementally refines a rectangular partition of the design

To simplify the mathematics and avoid oversensitivity to dimensions with larger domains, DIRECT first normalizes the search space to be the unit hypercube.

If we are minimizing $f(\mathbf{x})$ in the interval between lower and upper ranges ($\mathbf{a}$) and ($\mathbf{b}$) DIRECT will instead minimize:
If we are minimizing $f(x)$ in the interval between lower and upper ranges $a$ and $b$ DIRECT will instead minimize:

```math
g(\mathbf{x}) = f(\mathbf{x} \odot (\mathbf{b} - \mathbf{a}) + \mathbf{a})
```

After finding the minimum $f(\mathbf{x})$ of ($g$), the minimum of ($f$) is:
After finding the minimum $f(x)$ of $g$, the minimum of $f$ is:

```math
\mathbf{x}^* \odot (\mathbf{b} - \mathbf{a}) + \mathbf{a}
Expand All @@ -26,79 +26,65 @@ After finding the minimum $f(\mathbf{x})$ of ($g$), the minimum of ($f$) is:
## Partitioning the Search Space:
DIRECT maintains a partition of this unit hypercube into hyperrectangular intervals. Each interval has:

- A center ($c^{(i)}$)
- An associated objective function value $f(\mathbf{c}^{(i)})$
- A radius ($r^{(i)}$), which is the distance from the center to a vertex.
- A center $c^{(i)}$
- An associated objective function value $f(c^{(i)})$
- A radius $r^{(i)}$, which is the distance from the center to a vertex.

DIRECT begins every iteration by identifying intervals to be split with additional function evaluations. It splits the intervals for which a Lipschitz constant ($\ell$) exists such that that interval contains the Lipschitz lowerbound.
DIRECT begins every iteration by identifying intervals to be split with additional function evaluations. It splits the intervals for which a Lipschitz constant $\ell$ exists such that that interval contains the Lipschitz lowerbound.

---

## Splitting Intervals

DIRECT splits the identified intervals along their longest dimensions, dividing each interval into thirds. These splits refine the resolution in areas of interest, increasing accuracy near the minimum while avoiding excessive evaluations in less promising regions. The heuristic ensures that intervals are split in a way that balances the need for exploring the global space and refining local areas of interest.

### Visualizations:
---

## Lipschitz Lower Bound

The Lipschitz lower bound for an interval is a circular cone extending downwards from its center ($\mathbf{c}^{(i)}$):
![Image 17](assets/page_17.svg)

```math
f(\mathbf{x}) \geq f(\mathbf{c}^{(i)}) - \ell \|\mathbf{x} - \mathbf{c}^{(i)}\|_2
```
The figure above shows the progression of the univariate DIRECT method on this objective function, with intervals chosen for splitting rendered in blue.

This lower bound helps guide the decision of which intervals to split. The intervals for which a Lipschitz constant ($\ell$) exists such that the interval contains the Lipschitz lower bound are selected for further subdivision.
![Image 18](assets/page_18.svg)

---
Interval splitting in multiple dimensions for DIRECT requires choosing an ordering for the split dimensions.

## Visulizations:
To better understand the optimization process, here are some visualizations that represent different stages and aspects of the DIRECT algorithm's progress:
![Image 21](assets/page_21.svg)

![Image 13](assets/page_13.svg)
The potentially optimal intervals for the DIRECT method form a piecewise boundary that encloses all intervals along the lower-right. Each dot corresponds to a hyper-rectangle.

The Lipschitz lower bound for different Lipschitz constants. Not only does the estimated minimum change locally as the Lipschitz constant is varied, the region in which the minimum lies can vary as well.
![Image 20](assets/page_20.svg)

Potentially-optimal hyper-rectangle identification for a particular Lipschitz constant. Black dots represent DIRECT hyper-rectangles and their location in $f(c,r)$ space. A black line of slope is drawn through the dot belonging to the best interval. The dots for all other hyper-rectangles
must lie on or above this line.

---

![Image 14](assets/page_14.svg)
## Lipschitz Lower Bound

The DIRECT lower bound for different Lipschitz constants `. The lower bound is not continuous. The minimum does not change locally but can change regionally as the Lipschitz constant changes
The Lipschitz lower bound for an interval is a circular cone extending downwards from its center $c^{(i)}$:

---
```math
f(\mathbf{x}) \geq f(\mathbf{c}^{(i)}) - \ell \|\mathbf{x} - \mathbf{c}^{(i)}\|_2
```

This lower bound helps guide the decision of which intervals to split. The intervals for which a Lipschitz constant $\ell$ exists such that the interval contains the Lipschitz lower bound are selected for further subdivision.

### Visualizations:
---
![Image 12](assets/page_12.svg)

The left contour plot shows such a Lipschitz lower bound using five function evaluations. The right contour plot shows the approximation made by DIRECT, which divides the region into hyper-rectangles—one centered
about each design point. Making this assumption allows for the rapid calculation of the minimum of the lower bound.

---


![Image 21](assets/page_21.svg)

The potentially optimal intervals for the DIRECT method form a piecewise boundary that encloses all intervals along the lower-right. Each dot corresponds to a hyper-rectangle.

![Image 13](assets/page_13.svg)

---

![Image 20](assets/page_20.svg)

Potentially-optimal hyper-rectangle identification for a particular Lipschitz constant. Black dots represent DIRECT hyper-rectangles and their location in $(f(\mathbf{c}), r)$ space. A black line of slope is drawn through the dot belonging to the best interval. The dots for all other hyper-rectangles
must lie on or above this line.

---

![Image 18](assets/page_18.svg)
The Lipschitz lower bound for different Lipschitz constants. Not only does the estimated minimum change locally as the Lipschitz constant is varied, the region in which the minimum lies can vary as well.

Interval splitting in multiple dimensions (using DIRECT) requires choosing an ordering for the split dimensions.

---

![Image 14](assets/page_14.svg)

![Image 17](assets/page_17.svg)
The DIRECT lower bound for different Lipschitz constants `. The lower bound is not continuous. The minimum does not change locally but can change regionally as the Lipschitz constant changes

Consider the function $f(x) = sin(x) + sin(2x) + sin(4x) + sin(8x)$ on the interval [−2, 2] with a global minimizer near −0.272. Optimization is difficult because of multiple local minima. The figure above shows the progression of the univariate DIRECT method on this objective function, with intervals chosen for splitting rendered in blue.

---

0 comments on commit edc34ae

Please sign in to comment.