You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@leftwo and I are getting Omicron running on the dogfood rack. We're currently on BRM42220070. I made a quick update to Omicron using this patch. We can't run omicron-package uninstall, since that currently destroys the cxgbe{0,1} links and IP addresses that we currently need to log in to the machine. We instead rebuilt, and then ran OMICRON_NO_UNINSTALL=1 omicron-package install. That unpacks the tarballs into the correct place, and then calls svccfg import with the sled-agent manifest. We see this in the logfile:
[ Feb 21 18:04:31 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
[ Feb 21 18:04:31 Method "start" exited with status 0. ]
[ Feb 21 18:04:31 Rereading configuration. ]
[ Feb 21 18:04:31 No 'refresh' method defined. Treating as :true. ]
That's because svccfg import only updates any changed SMF properties and does not actually do anything like svcadm restart. So Alan and I ran svcadm restart manually. At this point, the sled agent started up. One of the first things it currently does is destroy any existing Oxide zones, VNICs and IP addresses. (Changing this to be more idempotent is tracked in #724.) It then recreates those objects. We then see this in the logs:
[ Feb 21 20:43:37 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
[ Feb 21 20:43:37 Method "start" exited with status 0. ]
note: configured to log to "/dev/stdout"
{"msg":"Starting mg-ddm service","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:37.71446934Z","hostname":"BRM42220070","pid":5581}
{"msg":"Importing mg-ddm service","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:37.714666334Z","hostname":"BRM42220070","pid":5581,"path":"/opt/oxide/mg-ddm/pkg/ddm/manifest.xml"}
{"msg":"Setting mg-ddm interfaces","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:37.858431628Z","hostname":"BRM42220070","pid":5581,"interfaces":"(\"cxgbe0/ll\" \"cxgbe1/ll\")"}
{"msg":"Enabling mg-ddm service","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:37.87371669Z","hostname":"BRM42220070","pid":5581}
{"msg":"detecting (real or simulated) SP","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:37.887126627Z","hostname":"BRM42220070","pid":5581}
{"msg":"setting up bootstrap agent server","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:37.887178889Z","hostname":"BRM42220070","pid":5581}
{"msg":"Ensuring config directory exists","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:37.910916301Z","hostname":"BRM42220070","pid":5581,"path":"/var/oxide"}
{"msg":"Sending prefix to ddmd for advertisement","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:37.972234806Z","hostname":"BRM42220070","pid":5581,"prefix":"Ipv6Prefix { addr: fdb0:a840:2504:154::, len: 64 }"}
{"msg":"Deleting existing zone","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:38.022867976Z","hostname":"BRM42220070","pid":5581,"zone_name":"oxz_nexus"}
{"msg":"Deleting existing zone","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:38.03022564Z","hostname":"BRM42220070","pid":5581,"zone_name":"oxz_oximeter"}
{"msg":"Deleting existing zone","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:38.053297124Z","hostname":"BRM42220070","pid":5581,"zone_name":"oxz_internal_dns"}
{"msg":"Deleting existing zone","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:38.061278113Z","hostname":"BRM42220070","pid":5581,"zone_name":"oxz_crucible_pantry"}
{"msg":"halt_and_remove_logged: Previous zone state: Running","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:40.395632247Z","hostname":"BRM42220070","pid":5581}
{"msg":"halt_and_remove_logged: Previous zone state: Running","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:40.421049899Z","hostname":"BRM42220070","pid":5581}
{"msg":"halt_and_remove_logged: Previous zone state: Running","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:40.461762884Z","hostname":"BRM42220070","pid":5581}
{"msg":"halt_and_remove_logged: Previous zone state: Running","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:40.483034618Z","hostname":"BRM42220070","pid":5581}
{"msg":"Deleting existing VNIC","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:40.50254981Z","hostname":"BRM42220070","pid":5581,"vnic_kind":"OxideControlVnic","vnic_name":"oxControlPublic0"}
{"msg":"Deleting existing VNIC","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:40.509786354Z","hostname":"BRM42220070","pid":5581,"vnic_kind":"OxideControlVnic","vnic_name":"oxControlService0"}
{"msg":"Deleting existing VNIC","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:40.509865746Z","hostname":"BRM42220070","pid":5581,"vnic_kind":"OxideControlVnic","vnic_name":"oxControlService1"}
{"msg":"Deleting existing VNIC","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:40.509884546Z","hostname":"BRM42220070","pid":5581,"vnic_kind":"OxideControlVnic","vnic_name":"oxControlService2"}
{"msg":"Deleting existing VNIC","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:40.509899616Z","hostname":"BRM42220070","pid":5581,"vnic_kind":"OxideControlVnic","vnic_name":"oxControlService3"}
{"msg":"Bootstrap Agent monitoring for hardware","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:40.543623717Z","hostname":"BRM42220070","pid":5581}
{"msg":"Creating HardwareManager","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:40.543672318Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@0,0/pci1022,1483@1,3/pci1344,3100@0/blkdev@w00A0750132753688,0\", dev_path: None }, slot: 17, variant: M2, identity: DiskIdentity { vendor: \"1344\", serial: \"214132753688\", model: \"Micron_7300_MTFDHBG1T9TDF\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.389984489Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@ab,0/pci1022,1483@1,3/pci1b96,0@0/blkdev@w0014EE81000D2E4D,0\", dev_path: Some(\"/dev/dsk/c11t0014EE81000D2E4Dd0\") }, slot: 2, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A633\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.39002444Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@0,0/pci1022,1483@3,3/pci1b96,0@0/blkdev@w0014EE81000D2EEA,0\", dev_path: Some(\"/dev/dsk/c3t0014EE81000D2EEAd0\") }, slot: 5, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A6C4\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.39003365Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@38,0/pci1022,1483@1,2/pci1b96,0@0/blkdev@w0014EE81000D2FFA,0\", dev_path: Some(\"/dev/dsk/c6t0014EE81000D2FFAd0\") }, slot: 8, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A6F4\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.390043161Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@38,0/pci1022,1483@1,1/pci1b96,0@0/blkdev@w0014EE81000D2EE9,0\", dev_path: Some(\"/dev/dsk/c5t0014EE81000D2EE9d0\") }, slot: 9, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A6BF\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.390055441Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@0,0/pci1022,1483@3,2/pci1b96,0@0/blkdev@w0014EE81000D2E9D,0\", dev_path: Some(\"/dev/dsk/c2t0014EE81000D2E9Dd0\") }, slot: 6, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A6D3\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.390089592Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@ab,0/pci1022,1483@1,4/pci1b96,0@0/blkdev@w0014EE81000D2E4A,0\", dev_path: Some(\"/dev/dsk/c12t0014EE81000D2E4Ad0\") }, slot: 3, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A630\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.390098322Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@0,0/pci1022,1483@3,4/pci1b96,0@0/blkdev@w0014EE81000D3035,0\", dev_path: Some(\"/dev/dsk/c4t0014EE81000D3035d0\") }, slot: 4, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A5CB\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.390108312Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@ab,0/pci1022,1483@1,2/pci1b96,0@0/blkdev@w0014EE81000D2E50,0\", dev_path: Some(\"/dev/dsk/c10t0014EE81000D2E50d0\") }, slot: 1, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A636\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.390116552Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@38,0/pci1022,1483@1,3/pci1b96,0@0/blkdev@w0014EE81000D2F5B,0\", dev_path: Some(\"/dev/dsk/c7t0014EE81000D2F5Bd0\") }, slot: 7, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A771\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.390125182Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@ab,0/pci1022,1483@1,1/pci1b96,0@0/blkdev@w0014EE81000D307F,0\", dev_path: Some(\"/dev/dsk/c9t0014EE81000D307Fd0\") }, slot: 0, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A5CD\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.390132393Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@38,0/pci1022,1483@3,3/pci1344,3100@0/blkdev@w00A0750132753657,0\", dev_path: None }, slot: 18, variant: M2, identity: DiskIdentity { vendor: \"1344\", serial: \"214132753657\", model: \"Micron_7300_MTFDHBG1T9TDF\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.390139583Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Sled already configured, loading sled agent","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.395595777Z","hostname":"BRM42220070","pid":5581,"component":"BootstrapAgent"}
{"msg":"Monitoring for hardware updates","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.395672679Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager","component":"BootstrapAgent"}
{"msg":"Performing full hardware scan","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.395691339Z","hostname":"BRM42220070","pid":5581,"component":"BootstrapAgent"}
{"msg":"Disabling switch zone (already complete)","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.39569994Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Loading Sled Agent: SledAgentRequest { id: 9a721086-36f5-4a39-af55-1c389fe5eaae, rack_id: 92bbdfed-f133-4489-a918-5e7f281b8760, gateway: Gateway { address: None, mac: MacAddr6([0, 13, 185, 84, 254, 228]) }, subnet: Ipv6Subnet { net: Ipv6Net(Ipv6Network { addr: fd00:1122:3344:101::, prefix: 64 }) } }","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.395849953Z","hostname":"BRM42220070","pid":5581,"component":"BootstrapAgent"}
{"msg":"setting up sled agent server","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.395943575Z","hostname":"BRM42220070","pid":5581}
{"msg":"created sled agent","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.396205362Z","hostname":"BRM42220070","pid":5581,"sled_id":"9a721086-36f5-4a39-af55-1c389fe5eaae","component":"SledAgent"}
{"msg":"xde driver configuration file appears to already use external IP workaround","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.468164956Z","hostname":"BRM42220070","pid":5581,"sled_id":"9a721086-36f5-4a39-af55-1c389fe5eaae","component":"SledAgent","conf_file":"\"/kernel/drv/xde.conf\""}
{"msg":"using '[AddrObject { interface: \"cxgbe0\", name: \"ll\" }, AddrObject { interface: \"cxgbe1\", name: \"ll\" }]' as data links for xde driver","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.704353522Z","hostname":"BRM42220070","pid":5581,"sled_id":"9a721086-36f5-4a39-af55-1c389fe5eaae","component":"SledAgent"}
{"msg":"Sled Agent upserting zpool to Storage Manager: oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.86118484Z","hostname":"BRM42220070","pid":5581,"sled_id":"9a721086-36f5-4a39-af55-1c389fe5eaae","component":"SledAgent"}
{"msg":"Inserting zpool: ZpoolName(d462a7f7-b628-40fe-80ff-4e4189e2d62b)","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.861257091Z","hostname":"BRM42220070","pid":5581,"component":"StorageManager"}
{"msg":"Sled Agent upserting zpool to Storage Manager: oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.861272012Z","hostname":"BRM42220070","pid":5581,"sled_id":"9a721086-36f5-4a39-af55-1c389fe5eaae","component":"SledAgent"}
{"msg":"Inserting zpool: ZpoolName(e4b4dc87-ab46-49fb-a4b4-d361ae214c03)","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.861286562Z","hostname":"BRM42220070","pid":5581,"component":"StorageManager"}
{"msg":"Sled Agent upserting zpool to Storage Manager: oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.861319323Z","hostname":"BRM42220070","pid":5581,"sled_id":"9a721086-36f5-4a39-af55-1c389fe5eaae","component":"SledAgent"}
{"msg":"Inserting zpool: ZpoolName(f4b4dc87-ab46-49fb-a4b4-d361ae214c03)","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.861331233Z","hostname":"BRM42220070","pid":5581,"component":"StorageManager"}
{"msg":"StorageWorker encountered unexpected error: Failed to get info for zpool 'oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b': Zpool execution error: Command [list -Hpo name,size,allocated,free,health oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b] executed and failed with status: exit status: 1 stdout: stderr: cannot open 'oxp_d462a7f7-b628-40fe-80ff-4e4189e2d62b': no such pool\n","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:41.878410055Z","hostname":"BRM42220070","pid":5581,"component":"StorageManager"}
{"msg":"Creating HardwareManager","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:41.893014421Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"StorageWorker encountered unexpected error: Failed to get info for zpool 'oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03': Zpool execution error: Command [list -Hpo name,size,allocated,free,health oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03] executed and failed with status: exit status: 1 stdout: stderr: cannot open 'oxp_e4b4dc87-ab46-49fb-a4b4-d361ae214c03': no such pool\n","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:41.894164879Z","hostname":"BRM42220070","pid":5581,"component":"StorageManager"}
{"msg":"StorageWorker encountered unexpected error: Failed to get info for zpool 'oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03': Zpool execution error: Command [list -Hpo name,size,allocated,free,health oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03] executed and failed with status: exit status: 1 stdout: stderr: cannot open 'oxp_f4b4dc87-ab46-49fb-a4b4-d361ae214c03': no such pool\n","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:43:41.933579192Z","hostname":"BRM42220070","pid":5581,"component":"StorageManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@ab,0/pci1022,1483@1,4/pci1b96,0@0/blkdev@w0014EE81000D2E4A,0\", dev_path: Some(\"/dev/dsk/c12t0014EE81000D2E4Ad0\") }, slot: 3, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A630\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.25421274Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@ab,0/pci1022,1483@1,2/pci1b96,0@0/blkdev@w0014EE81000D2E50,0\", dev_path: Some(\"/dev/dsk/c10t0014EE81000D2E50d0\") }, slot: 1, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A636\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254267031Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@0,0/pci1022,1483@3,2/pci1b96,0@0/blkdev@w0014EE81000D2E9D,0\", dev_path: Some(\"/dev/dsk/c2t0014EE81000D2E9Dd0\") }, slot: 6, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A6D3\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254283561Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@38,0/pci1022,1483@1,2/pci1b96,0@0/blkdev@w0014EE81000D2FFA,0\", dev_path: Some(\"/dev/dsk/c6t0014EE81000D2FFAd0\") }, slot: 8, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A6F4\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254299292Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@0,0/pci1022,1483@3,3/pci1b96,0@0/blkdev@w0014EE81000D2EEA,0\", dev_path: Some(\"/dev/dsk/c3t0014EE81000D2EEAd0\") }, slot: 5, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A6C4\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254346743Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@38,0/pci1022,1483@1,1/pci1b96,0@0/blkdev@w0014EE81000D2EE9,0\", dev_path: Some(\"/dev/dsk/c5t0014EE81000D2EE9d0\") }, slot: 9, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A6BF\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254406654Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@ab,0/pci1022,1483@1,1/pci1b96,0@0/blkdev@w0014EE81000D307F,0\", dev_path: Some(\"/dev/dsk/c9t0014EE81000D307Fd0\") }, slot: 0, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A5CD\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254424135Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@0,0/pci1022,1483@3,4/pci1b96,0@0/blkdev@w0014EE81000D3035,0\", dev_path: Some(\"/dev/dsk/c4t0014EE81000D3035d0\") }, slot: 4, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A5CB\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254435935Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@0,0/pci1022,1483@1,3/pci1344,3100@0/blkdev@w00A0750132753688,0\", dev_path: None }, slot: 17, variant: M2, identity: DiskIdentity { vendor: \"1344\", serial: \"214132753688\", model: \"Micron_7300_MTFDHBG1T9TDF\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254456935Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@ab,0/pci1022,1483@1,3/pci1b96,0@0/blkdev@w0014EE81000D2E4D,0\", dev_path: Some(\"/dev/dsk/c11t0014EE81000D2E4Dd0\") }, slot: 2, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A633\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254474006Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@38,0/pci1022,1483@3,3/pci1344,3100@0/blkdev@w00A0750132753657,0\", dev_path: None }, slot: 18, variant: M2, identity: DiskIdentity { vendor: \"1344\", serial: \"214132753657\", model: \"Micron_7300_MTFDHBG1T9TDF\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254485436Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Update from polling device tree: DiskAdded(UnparsedDisk { paths: DiskPaths { devfs_path: \"/devices/pci@38,0/pci1022,1483@1,3/pci1b96,0@0/blkdev@w0014EE81000D2F5B,0\", dev_path: Some(\"/dev/dsk/c7t0014EE81000D2F5Bd0\") }, slot: 7, variant: U2, identity: DiskIdentity { vendor: \"1b96\", serial: \"A084A771\", model: \"WUS4C6432DSP3X3\" } })","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254496796Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"Sled services found at /var/oxide/service.toml; loading","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254507587Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring service zone is initialized: Nexus","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254759493Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Service zone nexus does not yet exist","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.254780753Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Configuring new Omicron zone: oxz_nexus","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.319566208Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
ZZZ install_ with ["/opt/oxide/nexus.tar.gz"]
{"msg":"Installing Omicron zone: oxz_nexus","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:43.344921599Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Zone booting","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:45.729978643Z","hostname":"BRM42220070","pid":5581,"zone":"oxz_nexus","component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring address fd00:1122:3344:101::3 exists","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:48.803528129Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::3, prefix: 64 }))","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:48.803603261Z","hostname":"BRM42220070","pid":5581,"zone":"oxz_nexus","component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring address fd00:1122:3344:101::3 exists - OK","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:49.394285187Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"GZ addresses: []","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:49.394340609Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Setting up Nexus service","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:49.972534764Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Adding address: Static(V4(Ipv4Network { addr: 192.168.1.20, prefix: 24 }))","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:49.972599276Z","hostname":"BRM42220070","pid":5581,"zone":"oxz_nexus","component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring service zone is initialized: Oximeter","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:50.738549412Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Service zone oximeter does not yet exist","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:50.738592923Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Configuring new Omicron zone: oxz_oximeter","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:50.762946735Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
ZZZ install_ with ["/opt/oxide/oximeter.tar.gz"]
{"msg":"Installing Omicron zone: oxz_oximeter","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:50.791638794Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Zone booting","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:52.092392896Z","hostname":"BRM42220070","pid":5581,"zone":"oxz_oximeter","component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring address fd00:1122:3344:101::4 exists","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:55.006030213Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::4, prefix: 64 }))","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:55.006136496Z","hostname":"BRM42220070","pid":5581,"zone":"oxz_oximeter","component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring address fd00:1122:3344:101::4 exists - OK","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:55.642838207Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"GZ addresses: []","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:55.64289669Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Setting up oximeter service","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:55.93367823Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring service zone is initialized: InternalDNS","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:56.085815117Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Service zone internal_dns does not yet exist","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:56.085858328Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Configuring new Omicron zone: oxz_internal_dns","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:56.212388006Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
ZZZ install_ with ["/opt/oxide/internal_dns.tar.gz"]
{"msg":"Installing Omicron zone: oxz_internal_dns","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:56.242181795Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Zone booting","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:57.635230132Z","hostname":"BRM42220070","pid":5581,"zone":"oxz_internal_dns","component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring address fd00:1122:3344:1::1 exists","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:59.541230858Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:1::1, prefix: 64 }))","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:43:59.541371491Z","hostname":"BRM42220070","pid":5581,"zone":"oxz_internal_dns","component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring address fd00:1122:3344:1::1 exists - OK","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:00.228835358Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"GZ addresses: [\n fd00:1122:3344:1::2,\n]","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:00.228916899Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring GZ address fd00:1122:3344:1::2 exists","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:00.22894016Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Sending prefix to ddmd for advertisement","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:00.259054233Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent","prefix":"Ipv6Prefix { addr: fd00:1122:3344:1::, len: 64 }"}
G{"msg":"Setting up internal-dns service","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:00.570799179Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring service zone is initialized: CruciblePantry","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:00.72532663Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Service zone crucible_pantry does not yet exist","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:00.725377861Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Configuring new Omicron zone: oxz_crucible_pantry","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:00.752504004Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
ZZZ install_ with ["/opt/oxide/crucible_pantry.tar.gz"]
{"msg":"Installing Omicron zone: oxz_crucible_pantry","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:00.777960127Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Zone booting","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:02.089271576Z","hostname":"BRM42220070","pid":5581,"zone":"oxz_crucible_pantry","component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring address fd00:1122:3344:101::10 exists","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:04.195027841Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::10, prefix: 64 }))","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:04.195101303Z","hostname":"BRM42220070","pid":5581,"zone":"oxz_crucible_pantry","component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Ensuring address fd00:1122:3344:101::10 exists - OK","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:04.803763042Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"GZ addresses: []","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:04.803852195Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Setting up Crucible pantry service","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.286935862Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"Monitoring for hardware updates","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.406143844Z","hostname":"BRM42220070","pid":5581,"component":"HardwareManager"}
{"msg":"contacting server nexus, registering sled","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.406210105Z","hostname":"BRM42220070","pid":5581,"sled_id":"9a721086-36f5-4a39-af55-1c389fe5eaae","component":"SledAgent","baseboard":"Baseboard { identifier: \"BRM42220070\", model: \"913-0000019\", revision: 6 }","id":"9a721086-36f5-4a39-af55-1c389fe5eaae"}
{"msg":"Performing full hardware scan","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.406245706Z","hostname":"BRM42220070","pid":5581,"sled_id":"9a721086-36f5-4a39-af55-1c389fe5eaae","component":"SledAgent"}
{"msg":"Disabling switch zone (already complete)","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.40631664Z","hostname":"BRM42220070","pid":5581,"component":"ServiceManager","component":"BootstrapAgent"}
{"msg":"listening","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.406724367Z","hostname":"BRM42220070","pid":5581,"local_addr":"[fd00:1122:3344:101::1]:12345","component":"dropshot (SledAgent)"}
{"msg":"Sled Agent loaded; recording configuration","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.406784169Z","hostname":"BRM42220070","pid":5581,"component":"BootstrapAgent"}
{"msg":"Started listening","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.407146377Z","hostname":"BRM42220070","pid":5581,"component":"BootstrapAgentServer","local_addr":"[fdb0:a840:2504:154::1]:12346"}
{"msg":"Sending prefix to ddmd for advertisement","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.407166218Z","hostname":"BRM42220070","pid":5581,"prefix":"Ipv6Prefix { addr: fd00:1122:3344:101::, len: 64 }"}
{"msg":"bootstrap service initializing RSS","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.407179608Z","hostname":"BRM42220070","pid":5581,"component":"BootstrapAgent"}
{"msg":"Injecting RSS configuration: SetupServiceConfig {\n rack_subnet: fd00:1122:3344:100::,\n rack_secret_threshold: 1,\n gateway: Gateway {\n address: None,\n mac: MacAddr6(\n [\n 0,\n 13,\n 185,\n 84,\n 254,\n 228,\n ],\n ),\n },\n nexus_external_address: 192.168.1.20,\n}","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.40724846Z","hostname":"BRM42220070","pid":5581,"component":"RSS"}
{"msg":"RSS configuration looks like it has already been applied","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.40726625Z","hostname":"BRM42220070","pid":5581,"component":"RSS"}
{"msg":"RSS plan already created, loading from file","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.407298991Z","hostname":"BRM42220070","pid":5581,"component":"RSS"}
{"msg":"RSS plan already created, loading from file","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.407583908Z","hostname":"BRM42220070","pid":5581,"component":"RSS"}
{"msg":"Handing off control to Nexus","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.408045759Z","hostname":"BRM42220070","pid":5581,"component":"RSS"}
{"msg":"failed to notify nexus about sled agent: no record found for Query { name: Name(\"_nexus._tcp.control-plane.oxide.internal.\"), query_type: AAAA, query_class: IN }, will retry in 66.839276ms","v":0,"name":"SledAgent","level":40,"time":"2023-02-21T20:44:05.408313865Z","hostname":"BRM42220070","pid":5581,"sled_id":"9a721086-36f5-4a39-af55-1c389fe5eaae","component":"SledAgent"}
{"msg":"Disk at /devices/pci@0,0/pci1022,1483@3,3/pci1b96,0@0/blkdev@w0014EE81000D2EEA,0 already has a GPT","v":0,"name":"SledAgent","level":30,"time":"2023-02-21T20:44:05.40895173Z","hostname":"BRM42220070","pid":5581,"component":"StorageManager"}
thread 'tokio-runtime-worker' panicked at 'Failed to lookup IP: Resolve(ResolveError { kind: NoRecordsFound { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: AAAA, query_class: IN }, soa: None, negative_ttl: None, response_code: NXDomain, trusted: true } })', sled-agent/src/rack_setup/service.rs:451:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[ Feb 21 20:44:05 Stopping because all processes in service exited. ]
[ Feb 21 20:44:05 Executing stop method (:kill). ]
[ Feb 21 20:44:05 Restarting too quickly, changing state to maintenance. ]
You can see the sled agent load a service manifest from /var/oxide/service.toml. It then goes on to start Nexus, Oximeter, and the internal DNS service in zones. Towards the end, we then see some messages about loading an RSS plan from disk, and handing off to Nexus. We hit an unwrap at sled-agent/src/rack_setup/service.rs:451, where we're apparently looking up Nexus's IP by service name, and panicking if that fails.
I'm not sure what should happen here, or if Alan and I have put us in some unexpected situation by unpacking / uninstalling / reinstalling manually.
The text was updated successfully, but these errors were encountered:
@leftwo and I are getting Omicron running on the dogfood rack. We're currently on
BRM42220070
. I made a quick update to Omicron using this patch. We can't runomicron-package uninstall
, since that currently destroys thecxgbe{0,1}
links and IP addresses that we currently need to log in to the machine. We instead rebuilt, and then ranOMICRON_NO_UNINSTALL=1 omicron-package install
. That unpacks the tarballs into the correct place, and then callssvccfg import
with the sled-agent manifest. We see this in the logfile:That's because
svccfg import
only updates any changed SMF properties and does not actually do anything likesvcadm restart
. So Alan and I ransvcadm restart
manually. At this point, the sled agent started up. One of the first things it currently does is destroy any existing Oxide zones, VNICs and IP addresses. (Changing this to be more idempotent is tracked in #724.) It then recreates those objects. We then see this in the logs:You can see the sled agent load a service manifest from
/var/oxide/service.toml
. It then goes on to start Nexus, Oximeter, and the internal DNS service in zones. Towards the end, we then see some messages about loading an RSS plan from disk, and handing off to Nexus. We hit an unwrap atsled-agent/src/rack_setup/service.rs:451
, where we're apparently looking up Nexus's IP by service name, and panicking if that fails.I'm not sure what should happen here, or if Alan and I have put us in some unexpected situation by unpacking / uninstalling / reinstalling manually.
The text was updated successfully, but these errors were encountered: