`invdes` validators #1992

yaugenst-flex · 2024-10-02T12:04:33Z

Adds a bunch of validators for invdes as well as constraints on some fields.
Depends on #1988

tylerflex

looks good, thanks. minor suggestions

tidy3d/plugins/autograd/functions.py

tidy3d/plugins/expressions/metrics.py

tidy3d/plugins/invdes/design.py

tidy3d/plugins/invdes/initialization.py

tidy3d/plugins/invdes/region.py

tylerflex · 2024-10-02T13:48:38Z

also note seemes that #1988 is included in this?

yaugenst-flex · 2024-10-02T14:00:42Z

also note seemes that #1988 is included in this?

it is because this is branched off of that but has develop as a target. will be gone when that PR is merged and this is one rebased

e-g-melo

Thanks @yaugenst-flex! The new validators seem very detailed. Tomorrow, I will checkout your branch to run a real example.

e-g-melo · 2024-10-03T10:35:42Z

@yaugenst-flex, I ran the inverse design plugin example and got these errors in design_region.to_structure() and inv_des.to_simulation() when trying to call them without params0.

Should we make param0 optional in design_region.to_structure(), inv_des.to_simulation(), and inv_des.to_simulation_data()?

tidy3d/plugins/invdes/region.py

yaugenst-flex · 2024-10-03T10:47:31Z

Should we make param0 optional in design_region.to_structure(), inv_des.to_simulation(), and inv_des.to_simulation_data()?

hm good point, although it would only work for the initial parameters right? so if you want to give e.g. the optimized parameters, you still need to pass them explicitly. might be a bit confusing?

e-g-melo · 2024-10-03T11:14:23Z

hm good point, although it would only work for the initial parameters right? so if you want to give e.g. the optimized parameters, you still need to pass them explicitly. might be a bit confusing?

Ok.. in that case, maybe we can simply modify the example and use design_region.to_structure(design_region.initial_parameters)

e-g-melo · 2024-10-03T11:22:33Z

@yaugenst-flex, I made the filter radius larger than the z direction and set uniform=(False, False, False). Then, I got this validation error. Is it the expected validation error? If that is the case, would it be possible to include a message that explicitly informs the user that the filter radius is larger than the design region in the z direction?

e-g-melo · 2024-10-03T11:38:49Z

In the case we set uniform=(False, False, True) and set the projection filter radius larger than the x, y dimensions, this validation error is shown. Would it be possible to have a message that tells the user which component to change to solve the issue? For example:

ERROR: Could not evaluate transformations: Conic filter radius larger than the
design region is not supported. Padding widths: [240 240], Array
shape: (240, 252). Ensure that the conic filter radius is smaller than
the corresponding dimension of the design region.

e-g-melo · 2024-10-03T13:49:35Z

I got this error in the 2nd iteration step when trying multiple frequencies in the metrics monitor but selecting a single one in metrics. Is it expected to work on this PR?

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[27], [line 1](vscode-notebook-cell:?execution_count=27&line=1)
----> [1](vscode-notebook-cell:?execution_count=27&line=1) result = optimizer.run()

File /mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:132, in AbstractOptimizer.run(self, post_process_fn, callback, params0)
    [117](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:117) """Run this inverse design problem from an optional initial set of parameters.
    [118](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:118) 
    [119](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:119) Parameters
   (...)
    [129](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:129)     Deprecated. Initial set of parameters. Use ``TopologyDesignRegion.intialization_spec`` instead.
    [130](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:130) """
    [131](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:131) starting_result = self.initialize_result(params0)
--> [132](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:132) return self.continue_run(
    [133](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:133)     result=starting_result,
    [134](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:134)     num_steps=self.num_steps,
    [135](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:135)     post_process_fn=post_process_fn,
    [136](https://vscode-remote
[InverseDesign_Validators3D.zip](https://github.com/user-attachments/files/17245123/InverseDesign_Validators3D.zip)
+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:136)     callback=callback,
    [137](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:137) )

File /mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:180, in AbstractOptimizer.continue_run(self, result, num_steps, post_process_fn, callback)
    [178](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:178) for step_index in range(done_steps, done_steps + num_steps):
    [179](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:179)     aux_data = {}
--> [180](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:180)     val, grad = val_and_grad_fn(params, aux_data=aux_data)
    [182](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:182)     if anp.allclose(grad, 0.0):
    [183](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:183)         td.log.warning(
    [184](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:184)             "All elements of the gradient are almost zero. This likely indicates "
    [185](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:185)             "a problem with the optimization set up. This can occur if the symmetry of the "
   (...)
    [193](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:193)             "function are contributing to the return value."
    [194](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/plugins/invdes/optimizer.py:194)         )

File ~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/wrap_util.py:20, in unary_to_nary.<locals>.nary_operator.<locals>.nary_f(*args, **kwargs)
     [18](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/wrap_util.py:18) else:
     [19](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/wrap_util.py:19)     x = tuple(args[i] for i in argnum)
---> [20](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/wrap_util.py:20) return unary_operator(unary_f, x, *nary_op_args, **nary_op_kwargs)

File ~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/differential_operators.py:143, in value_and_grad(fun, x)
    [139](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/differential_operators.py:139) if not vspace(ans).size == 1:
    [140](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/differential_operators.py:140)     raise TypeError("value_and_grad only applies to real scalar-output "
    [141](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/differential_operators.py:141)                     "functions. Try jacobian, elementwise_grad or "
    [142](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/differential_operators.py:142)                     "holomorphic_grad.")
--> [143](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/differential_operators.py:143) return ans, vjp(vspace(ans).ones())

File ~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:14, in make_vjp.<locals>.vjp(g)
---> [14](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:14) def vjp(g): return backward_pass(g, end_node)

File ~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:21, in backward_pass(g, end_node)
     [19](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:19) for node in toposort(end_node):
     [20](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:20)     outgrad = outgrads.pop(node)
---> [21](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:21)     ingrads = node.vjp(outgrad[0])
     [22](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:22)     for parent, ingrad in zip(node.parents, ingrads):
     [23](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:23)         outgrads[parent] = add_outgrads(outgrads.get(parent), ingrad)

File ~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:67, in defvjp.<locals>.vjp_argnums.<locals>.<lambda>(g)
     [64](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:64)         raise NotImplementedError(
     [65](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:65)             "VJP of {} wrt argnum 0 not defined".format(fun.__name__))
     [66](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:66)     vjp = vjpfun(ans, *args, **kwargs)
---> [67](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:67)     return lambda g: (vjp(g),)
     [68](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:68) elif L == 2:
     [69](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/autograd/core.py:69)     argnum_0, argnum_1 = argnums

File /mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:594, in _run_bwd.<locals>.vjp(data_fields_vjp)
    [591](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:591) def vjp(data_fields_vjp: AutogradFieldMap) -> AutogradFieldMap:
    [592](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:592)     """dJ/d{sim.traced_fields()} as a function of Function of dJ/d{data.traced_fields()}"""
--> [594](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:594)     sim_adj, adjoint_source_info = setup_adj(
    [595](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:595)         data_fields_vjp=data_fields_vjp,
    [596](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:596)         sim_data_orig=sim_data_orig,
    [597](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:597)         sim_fields_keys=sim_fields_keys,
    [598](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:598)     )
    [600](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:600)     # run adjoint simulation
    [601](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:601)     task_name_adj = str(task_name) + "_adjoint"

File /mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:748, in setup_adj(data_fields_vjp, sim_data_orig, sim_fields_keys)
    [745](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:745) data_vjp_paths = set(data_fields_vjp.keys())
    [747](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:747) num_monitors = len(sim_data_orig.simulation.monitors)
--> [748](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:748) adjoint_monitors = sim_data_orig.simulation.with_adjoint_monitors(sim_fields_keys).monitors[
    [749](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:749)     num_monitors:
    [750](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:750) ]
    [752](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:752) sim_adj, adjoint_source_info = sim_data_vjp.make_adjoint_sim(
    [753](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:753)     data_vjp_paths=data_vjp_paths, adjoint_monitors=adjoint_monitors
    [754](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:754) )
    [756](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/web/api/autograd/autograd.py:756) if _INSPECT_ADJOINT_FIELDS:

File /mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/simulation.py:3633, in Simulation.with_adjoint_monitors(self, sim_fields_keys)
   [3631](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/simulation.py:3631) mnts_fld, mnts_eps = self.make_adjoint_monitors(structure_indices=structure_indices)
   [3632](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/simulation.py:3632) monitors = list(self.monitors) + list(mnts_fld) + list(mnts_eps)
-> [3633](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/simulation.py:3633) return self.copy(update=dict(monitors=monitors))

File /mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:223, in Tidy3dBaseModel.copy(self, deep, **kwargs)
    [221](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:221) kwargs.update(deep=deep)
    [222](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:222) new_copy = pydantic.BaseModel.copy(self, **kwargs)
--> [223](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:223) return self.validate(new_copy.dict())

File ~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:711, in BaseModel.validate(cls, value)
    [708](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:708) value = cls._enforce_dict_if_root(value)
    [710](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:710) if isinstance(value, dict):
--> [711](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:711)     return cls(**value)
    [712](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:712) elif cls.__config__.orm_mode:
    [713](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:713)     return cls.from_orm(value)

File /mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:146, in Tidy3dBaseModel.__init__(self, **kwargs)
    [144](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:144) """Init method, includes post-init validators."""
    [145](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:145) log.begin_capture()
--> [146](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:146) super().__init__(**kwargs)
    [147](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:147) self._post_init_validators()
    [148](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev/tidy3d/tidy3d/components/base.py:148) log.end_capture(self)

File ~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:341, in BaseModel.__init__(__pydantic_self__, **data)
    [339](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:339) values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
    [340](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:340) if validation_error:
--> [341](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:341)     raise validation_error
    [342](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:342) try:
    [343](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/Tidy3D_Dev_Test/test_serializable_ops/~/miniconda3/envs/tidy3d_dev/lib/python3.12/site-packages/pydantic/v1/main.py:343)     object_setattr(__pydantic_self__, '__dict__', values)

ValidationError: 1 validation error for Simulation
monitors
  'monitors' names are not unique, given ['MNT_0', 'MNT_1', 'MNT_2', 'adjoint_fld_4', 'adjoint_fld_4', 'adjoint_eps_4']. (type=value_error.setup)

e-g-melo · 2024-10-03T14:05:25Z

If I set length_scale to -1 I get this error, which is correct.

But if I set length_scale to 0, I get this other error. What should happen if length_scale=0? Should we validate it to >0 as in the conic filter radius?

e-g-melo · 2024-10-03T14:09:32Z

override_structure_dl=0 passed the validation

e-g-melo · 2024-10-03T14:34:29Z

learning_rate=0 passed the validation. Should we validate it for >0?

yaugenst-flex · 2024-10-03T16:24:57Z

Hey @e-g-melo, these are great! Thanks a lot for catching all of this. Point by point:

I made the filter radius larger than the z direction and set uniform=(False, False, False). Then, I got this validation error. Is it the expected validation error?

No this is not expected, it looks like the parameters have the wrong shape. I'll have to look into this one.

In the case we set uniform=(False, False, True) and set the projection filter radius larger than the x, y dimensions, this validation error is shown. Would it be possible to have a message that tells the user which component to change to solve the issue?

Ok I'll try to improve this. It's a bit tricky because scalar dimensions are removed before convolution, and the error is raised in the convolution, so there the information about the physical axis is lost. Maybe validation can happen a bit earlier.

I got this error in the 2nd iteration step when trying multiple frequencies in the metrics monitor but selecting a single one in metrics. Is it expected to work on this PR?

In principle this should work. It looks like the adjoint_fld_4 monitor is added twice, which causes simulation validation to fail. So I don't think this is a problem with the newly introduced validation. I'll investigate.

But if I set length_scale to 0, I get this other error. What should happen if length_scale=0? Should we validate it to >0 as in the conic filter radius?

Good catch, fixed. length_scale must be positive.

override_structure_dl=0 passed the validation

Oooh this is interesting, and confusing. That field is being validated according to typing.Union[pd.PositiveFloat, typing.Literal[False]], so I think override_structure_dl=0 is being coerced into False in the validation. which disables the override structure. This might actually be OK behavior? What do you think @tylerflex? To be fair we could also just change this to pd.NonNegativeFloat, I think this will lead to exactly the same behavior. False would be interpreted as 0 in that case.

learning_rate=0 passed the validation. Should we validate it for >0?

Absolutely, fixed.

tylerflex · 2024-10-03T17:03:04Z

That field is being validated according to typing.Union[pd.PositiveFloat, typing.Literal[False]], so I think override_structure_dl=0 is being coerced into False in the validation. which disables the override structure. This might actually be OK behavior? What do you think @tylerflex? To be fair we could also just change this to pd.NonNegativeFloat, I think this will lead to exactly the same behavior. False would be interpreted as 0 in that case.

seems like expected behavior to me, although a bit confusing. I'd think from the user perspective, changing to NonNegativeFloat would be exactly the same? So maybe we can just leave it as is?

tylerflex · 2024-10-03T17:43:31Z

something to note, we should update this to use the new initialization spec

tidy3d/tidy3d/plugins/invdes/optimizer.py

Line 184 in ce64857

"All elements of the gradient are almost zero. This likely indicates "

yaugenst-flex · 2024-10-04T13:40:06Z

Ok, I think everything is addressed now except for the issue with the duplicate adjoint field monitors, which I think we need to investigate further, but it is not related to validation.

tylerflex

Looks good overall, I still take some issue with the Metric.f thing, see comments. Basically I feel like it should operate roughly like xarray, so it's easy to take my existing postprocessing sel() code and transport it here, specitically

allow single f value, not just array.
if f is left out it automatically is selected (only works if single f in dataset because we dont support arrays).

Thoughts?

tidy3d/plugins/expressions/metrics.py

- Constraints for design parameters to be in range [0, 1] - Constraints on optimizer, penalty, transformation arguments such as `beta`, `eta`, `learning_rate` - Validation for forward and gradient evaluation of both transformations and penalties - Validation for metrics included in `Design` objects - Renamed `Metric.freqs` --> `Metric.f` - Make `Metric.f` optional

yaugenst-flex requested review from momchil-flex and tylerflex October 2, 2024 12:04

yaugenst-flex self-assigned this Oct 2, 2024

yaugenst-flex requested a review from e-g-melo October 2, 2024 13:41

tylerflex requested changes Oct 2, 2024

View reviewed changes

e-g-melo reviewed Oct 3, 2024

View reviewed changes

momchil-flex reviewed Oct 3, 2024

View reviewed changes

tidy3d/plugins/invdes/region.py Outdated Show resolved Hide resolved

momchil-flex approved these changes Oct 3, 2024

View reviewed changes

yaugenst-flex force-pushed the yaugenst-flex/invdes-validators branch 3 times, most recently from a51913a to 9798ef2 Compare October 4, 2024 12:57

yaugenst-flex requested a review from tylerflex October 4, 2024 13:40

tylerflex requested changes Oct 4, 2024

View reviewed changes

tidy3d/plugins/expressions/metrics.py Outdated Show resolved Hide resolved

yaugenst-flex requested a review from tylerflex October 7, 2024 08:13

feat: Support for arbitrary padding sizes in autograd plugin

ed618a9

yaugenst-flex added 2 commits October 7, 2024 11:23

feat: Method for extracting sub-components from nested expressions

41ab771

yaugenst-flex force-pushed the yaugenst-flex/invdes-validators branch from 946c607 to b89886c Compare October 7, 2024 09:35

momchil-flex merged commit 9fd7cf2 into develop Oct 7, 2024
15 checks passed

momchil-flex deleted the yaugenst-flex/invdes-validators branch October 7, 2024 10:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`invdes` validators #1992

`invdes` validators #1992

yaugenst-flex commented Oct 2, 2024

tylerflex left a comment

tylerflex commented Oct 2, 2024

yaugenst-flex commented Oct 2, 2024

e-g-melo left a comment

e-g-melo commented Oct 3, 2024

yaugenst-flex commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024 •

edited

Loading

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

yaugenst-flex commented Oct 3, 2024 •

edited

Loading

tylerflex commented Oct 3, 2024

tylerflex commented Oct 3, 2024

yaugenst-flex commented Oct 4, 2024

tylerflex left a comment

invdes validators #1992

invdes validators #1992

Conversation

yaugenst-flex commented Oct 2, 2024

tylerflex left a comment

Choose a reason for hiding this comment

tylerflex commented Oct 2, 2024

yaugenst-flex commented Oct 2, 2024

e-g-melo left a comment

Choose a reason for hiding this comment

e-g-melo commented Oct 3, 2024

yaugenst-flex commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024 • edited Loading

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

e-g-melo commented Oct 3, 2024

yaugenst-flex commented Oct 3, 2024 • edited Loading

tylerflex commented Oct 3, 2024

tylerflex commented Oct 3, 2024

yaugenst-flex commented Oct 4, 2024

tylerflex left a comment

Choose a reason for hiding this comment

`invdes` validators #1992

`invdes` validators #1992

e-g-melo commented Oct 3, 2024 •

edited

Loading

yaugenst-flex commented Oct 3, 2024 •

edited

Loading