Feature: Pluggable Payload Decoder in UI

[marvin-hansen]

Pluggable Payload Decoder with WebAssembly Support

Summary

Introduce support for pluggable payload decoders in the Web UI to enable decoding of binary protocols into JSON, leveraging WebAssembly (WASM) modules. This feature would allow users to provide custom decoders for payload inspection and visualization in human-readable formats.

---

Motivation

• User Use Case: Users working with binary protocols often need to inspect payloads in a structured, human-readable format. Currently, the Web UI lacks the ability to decode binary payloads beyond basic formats like Base64.
• Flexibility: Adding support for pluggable decoders would enable users to decode payloads using custom or specialized serialization protocols.
• Efficiency: Instead of introducing server-side changes, the feature allows decoders to operate entirely in the UI, leveraging WASM for performance and portability.
• Scalability: Decoders implemented in WASM can be packaged and reused across multiple applications.

---

Design Proposal

1. WASM-Based Pluggable Decoders

• Implement a mechanism to load and execute WebAssembly modules in the Web UI.
• Define a standard interface for decoder modules to interact with the Web UI. For example:

  interface Decoder {
    decode(payload: Uint8Array): string;
    metadata(): { name: string; description: string };
  }

2. Codec Identification

• Custom Codec Tag:
  Add support for a custom codec tag in message headers:
  • If a codec tag is present, the UI attempts to load the corresponding decoder (built-in or plugin).
  • If no codec is found, fallback behavior includes:
    • Showing an error message stating the missing codec.
    • Defaulting to displaying raw bytes.

---

3. Schema Registry

• Build a schema registry where users can define and manage schemas for serialization protocols (e.g., Protobuf, Avro, custom Rust structs).
• Optionally, allow users to paste definitions (e.g., Rust structs, C# classes) and compile them into WASM decoders via the UI.

---

4. Built-in Support for Common Formats

• Add built-in decoders for commonly used serialization formats:
  • JSON
  • TOML
  • YAML
  • Base64
  • XML
• Format decoded data for better readability in the Web UI.

---

Workflow Example

1. User uploads a WASM decoder module and registers it in the Web UI.
2. Messages with payloads encoded in supported formats or matching a codec tag are decoded using the corresponding module.
3. If no decoder is found, the raw payload is displayed, with an error indicating the need for a suitable codec.

---

Benefits

• Extensibility: Users can extend functionality by creating custom WASM decoders without modifying the core application.
• Standardization: WASM ensures cross-language compatibility and performance optimization.
• Improved Usability: Provides users with a powerful tool to visualize and debug payloads directly in the Web UI.

---

Potential Challenges

• Security: Ensure that WASM modules are sandboxed to prevent malicious behavior.
• Performance: Assess the impact of loading and running WASM decoders in real-time on the UI's responsiveness.
• Complexity: Balancing ease of use for basic users with extensibility for advanced users.

---

Alternatives Considered

• Server-side Decoding: Rejected due to additional complexity and lack of scalability.
• Fixed Format Decoding: Limited flexibility, unsuitable for diverse binary protocols.

---

Next Steps

1. Define the WASM decoder interface and integration points in the Web UI.
2. Implement a prototype with built-in decoders for JSON, Base64, and XML.
3. Gather user feedback on usability and extensibility.
4. Expand support to custom WASM modules and schema management.

---

References

• WebAssembly Documentation
• [Examples of Binary Protocols: Protobuf, Bincode, SBE, Avro]
• User-contributed feedback from discussion in discord.