Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Pluggable Discoveries #1333

Merged
merged 74 commits into from
Aug 23, 2021
Merged
Changes from 1 commit
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
386740b
[skip changelog] Add DiscoveryManager to PackageManager
silvanocerza Aug 9, 2021
ebca55f
Add loading of PluggableDiscoveries when loading a platform release
silvanocerza Jun 23, 2021
7cf3215
Added compatibility layer for non-pluggable platforms
cmaglie Jun 29, 2021
f442b1a
Implemented board list with discoveries
silvanocerza Aug 9, 2021
dd6a8fc
Implemented discovery loading after initialization
cmaglie Jun 29, 2021
f07775a
Implemented board watch with discoveries
silvanocerza Aug 9, 2021
e80869b
Fix load discoveries tests
silvanocerza Jun 29, 2021
23bab36
Fix some issues with board list watcher
silvanocerza Jun 30, 2021
3b4bd38
Fix FindToolsRequiredFromPlatformRelease not returning discoveries
silvanocerza Jun 30, 2021
47b84f5
Enhanced handling of some discoveries states
silvanocerza Jul 2, 2021
b32749f
Fix PackageManager reset
silvanocerza Jul 6, 2021
0d2add4
Add function to convert discovery.Port to rpc.Port
silvanocerza Aug 9, 2021
8bb84b2
Moved reference argument parsing to new package
silvanocerza Jul 6, 2021
3ff8287
Fix functions docstrings
silvanocerza Jul 8, 2021
ec3dfbc
Remove duplicated code to initialize Sketch path
silvanocerza Aug 9, 2021
fb934e5
Add property conversion for platform not supporting pluggable discovery
silvanocerza Jul 15, 2021
38c4325
Fix board list watch not working
silvanocerza Jul 15, 2021
a3007fe
Fix crash when converting Port to rpc struct
silvanocerza Jul 20, 2021
2d69063
Add generic Port argument
silvanocerza Jul 15, 2021
7e6dfca
Change gRPC upload functions to use new Port message
silvanocerza Aug 9, 2021
e6c33ea
Add support for upload user fields
silvanocerza Aug 9, 2021
4a586d9
Fix upload unit tests
silvanocerza Jul 20, 2021
67ae164
Fix code naming issues
silvanocerza Jul 20, 2021
ddf1603
Added builtin:mdns-discovery
cmaglie Jul 21, 2021
ea61dca
Do not panic if discovery tool is not installed
cmaglie Jul 21, 2021
d255c3d
Implemented port/protocol detection at CLI startup time
silvanocerza Aug 9, 2021
18777b2
Perform 1200bps-touch only on serial ports
cmaglie Jul 21, 2021
709db6f
Added missing properties for pluggable upload
cmaglie Jul 21, 2021
322ce23
Correctly implemented 'board list' timeout option
silvanocerza Aug 9, 2021
4fc7857
Updated mdns-discovery to 0.9.2
cmaglie Jul 22, 2021
77a40fd
Add documentation
silvanocerza Aug 10, 2021
f8ac21d
Add board properties to board list command and gRPC function
silvanocerza Aug 9, 2021
5ecf4d8
Fix documentation and code comments
silvanocerza Aug 10, 2021
c8fb6e4
Fix crash when attempting upload without specifying port address
silvanocerza Jul 23, 2021
2a7d3d2
Fix unit tests
silvanocerza Jul 23, 2021
015aea3
Update go-properties-orderedmap to fix discovery properties issues
silvanocerza Aug 11, 2021
33f8848
Fix more documentation
silvanocerza Jul 26, 2021
bd68d5e
Clarify pluggable discovery specification
silvanocerza Jul 26, 2021
69c6812
More documentation fixes
silvanocerza Jul 27, 2021
aec9695
Add upload_port properties docs in platform specification
silvanocerza Jul 27, 2021
ed318af
Change links from pluggable discovery RFC to official docs
silvanocerza Jul 27, 2021
95b69fa
Add more upload mock integration tests
silvanocerza Jul 27, 2021
3e3fcd2
Fix integration tests
silvanocerza Jul 28, 2021
b7d594f
Change property to declare pluggable discoveries
silvanocerza Aug 10, 2021
763ce6f
Change property to declare pluggable discoveries
silvanocerza Jul 28, 2021
ae3a497
Fix documentation
silvanocerza Jul 28, 2021
3f484ed
Fix loading of platform not supporting pluggable discovery
silvanocerza Jul 28, 2021
be9e879
Fix more documentation
silvanocerza Jul 29, 2021
225c123
Add pluggable discovery states documentation
silvanocerza Aug 2, 2021
569429d
Enhanced handling of pluggable discoveries states
silvanocerza Aug 9, 2021
a0f2aee
Discoveries processes are now killed if the HELLO command fails
silvanocerza Aug 3, 2021
2624785
Add pluggable discovery logging
silvanocerza Aug 4, 2021
2a555ff
Enhanced handling of failing pluggable discoveries
silvanocerza Aug 4, 2021
71dcc0e
Fix pluggable discoveries parallelization
silvanocerza Aug 9, 2021
a0a6371
Discoveries event channels are now created when start sync is called
silvanocerza Aug 9, 2021
cfc7aa4
Cached ports are now reset on discovery stop
silvanocerza Aug 9, 2021
2a34f7d
Renamed ListSync methods to ListCachedPorts
silvanocerza Aug 9, 2021
85607d6
Pluggable discovery upload user fields are now limited to 50 chars
silvanocerza Aug 9, 2021
a5be354
Fix i18n strings
silvanocerza Aug 9, 2021
e68e4e0
Fix failing integration tests
silvanocerza Aug 10, 2021
eef60aa
Fix i18n data
silvanocerza Aug 10, 2021
f8814bc
Fix integration tests again
silvanocerza Aug 10, 2021
5aa1f1d
[skip changelog] Internationalize strings added for pluggable discove…
per1234 Aug 10, 2021
9941402
Update docs/pluggable-discovery-specification.md
silvanocerza Aug 10, 2021
53e1d25
Fix failing workflows
silvanocerza Aug 11, 2021
9a355b7
Updated upload-mock tests for generation
cmaglie Aug 12, 2021
41db8b5
Added a lot of mock upload test (also with programmer option)
cmaglie Aug 16, 2021
b322874
test_upload_mock: Handle '{' and '}' in recipes
cmaglie Aug 16, 2021
d7a576c
network ota: autoconvert network_patter from legacy
cmaglie Aug 17, 2021
3c10109
Automatically add port detection properties for network discovery
cmaglie Aug 19, 2021
e6ee163
Slightly improved 'board list' text output
cmaglie Aug 20, 2021
75c7fef
Default 'board list' timeout to 1s
cmaglie Aug 20, 2021
c100e14
Added some code review fixes
cmaglie Aug 23, 2021
40943b1
Added unit test for legacy-package conversion to pluggable discovery
cmaglie Aug 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions arduino/cores/packagemanager/loader.go
Original file line number Diff line number Diff line change
Expand Up @@ -356,9 +356,62 @@ func (pm *PackageManager) loadPlatformRelease(platform *cores.PlatformRelease, p
return fmt.Errorf(tr("loading boards: %s"), err)
}

if !platform.PluggableDiscoveryAware {
convertLegacyPlatformToPluggableDiscovery(platform)
}
silvanocerza marked this conversation as resolved.
Show resolved Hide resolved
return nil
}

func convertLegacyPlatformToPluggableDiscovery(platform *cores.PlatformRelease) {
toolsProps := platform.Properties.SubTree("tools").FirstLevelOf()
for toolName, toolProps := range toolsProps {
if !toolProps.ContainsKey("upload.network_pattern") {
continue
}

// Convert network_pattern configuration to pluggable discovery
convertedToolName := toolName + "__pluggable_network"
convertedProps := convertLegacyNetworkPatternToPluggableDiscovery(toolProps, convertedToolName)

// Merge the converted properties in the root configuration
platform.Properties.Merge(convertedProps)

// Add the network upload to the boards using the old method
for _, board := range platform.Boards {
oldUploadTool := board.Properties.Get("upload.tool")
if oldUploadTool == toolName && !board.Properties.ContainsKey("upload.tool.network") {
board.Properties.Set("upload.tool.network", convertedToolName)
// fmt.Printf("ADDED: %s.upload.tool.network=%s\n", board, convertedToolName)
silvanocerza marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
}

func convertLegacyNetworkPatternToPluggableDiscovery(props *properties.Map, newToolName string) *properties.Map {
pattern, ok := props.GetOk("upload.network_pattern")
if !ok {
return nil
}
props.Remove("upload.network_pattern")
pattern = strings.ReplaceAll(pattern, "{serial.port}", "{upload.port.address}")
pattern = strings.ReplaceAll(pattern, "{network.port}", "{upload.port.properties.port}")
if strings.Contains(pattern, "{network.password}") {
props.Set("upload.field.password", "Password")
props.Set("upload.field.password.secret", "true")
pattern = strings.ReplaceAll(pattern, "{network.password}", "{upload.field.password}")
}
props.Set("upload.pattern", pattern)

prefix := "tools." + newToolName + "."
res := properties.NewMap()
for _, k := range props.Keys() {
v := props.Get(k)
res.Set(prefix+k, v)
// fmt.Println("ADDED:", prefix+k+"="+v)
silvanocerza marked this conversation as resolved.
Show resolved Hide resolved
}
return res
}

func (pm *PackageManager) loadProgrammer(programmerProperties *properties.Map) *cores.Programmer {
return &cores.Programmer{
Name: programmerProperties.Get("name"),
Expand Down