Questions about aspect filtering & destinations

[ThomasFreedman]

I'm trying to understand the intended purpose of destination aspects and how best to use them.

I understand using the app name as the first aspect, but it seems any additional levels tie the destination to only messages for those aspects the destination is constructed with.

If the design intention is to use multiple destinations for each type of message an app may require, defining aspects in the destination constructor makes sense.

I tend to think of destinations as being tied to the app, not specific functions of an app, so an app would only need one destination (or possibly 2, IN & OUT). If that was the intention, I would expect to see an aspect property and methods to get the aspects from incomming messages or from the RNS.Transport object so message handlers could process messages based on aspects (thinking of aspects as app sub-functions).

So I'm trying to decide if I need to setup separate destinations for:

1. making announcements
2. general communications
3. receiving message acknowledgements (proofs?)

I'm thinking my app needs 3 RNS destinations:
1 To announce availability of a new media item for publication (to anyone / the world)
An RNS annoounce is perfect for this, with the appData being an IPFS hash. Not sure
tho, as it looks like an announce is sent when the app is started automatically.
2 Receiving an acknowledgement from a publisher they have pinned the media in IPFS
3 Receiving a message that publishing to a specific platform has completed

3 RNS destinations would be required since they require different aspect filters, assuming the aspect filtering is static with destination definition.

[faragher]

I had similar questions and I can't really explain it myself. So I provide this: #260

From the response to my issue:
Every destination is named by a .-separated list of aspects. The first is always the "application name" (use it to group all destination types from a single application, system or protocol together). All aspects under that can be defined in any way you like, but a complete destination name, eg. myapplication.rpc.sensorstatus is always unique and distinct from all others, and for each Identity coupled to it, forms a completely unique address on the network.

My current understanding is that it's a name that acts as a filter. If you have Foo.Bar then you can use that as a destination and process announces, data, etc. in code. If you had Foo.Bar.Announce and Foo.Bar.Data, then you could expect the data coming in to be filtered to the proper destination.

I don't know why you'd do this, but what I don't know I can just about squeeze into the Grand Canyon, and more options are usually better than fewer.

[markqvist]

I tend to think of destinations as being tied to the app, not specific functions of an app, so an app would only need one destination (or possibly 2, IN & OUT)

Most simple apps will only need one destination, yes. As complexity grows, you can easily add more. It is a very useful expandability to have.

No permanent OUT destinations need to be made. All outgoing communication in Reticulum is source-less. They are as such an abstract construct, and are always ephemerally created and directly tied to a specific IN destination somewhere else in the network. So you just create one when you want to communicate with a destination somewhere on the network, and only when you want to be the initiator of the communication. If one piece of code only listens for incoming data or waits for a Link to be opened, it only needs an IN destination.

so message handlers could process messages based on aspects (thinking of aspects as app sub-functions).

In most cases, it is probably better and more efficient to place sub-functions under various paths in a Request handler. But you can of course also code something completely custom using just the raw packet API, or even use as many destinations as you wish. For efficiency over extremely low-bandwidth, use as few different destinations as makes sense, and make use of the built-in APIs for resource transfers, requests, and so on. They are designed to continue working in adverse conditions.

If the design intention is to use multiple destinations for each type of message an app may require, defining aspects in the destination constructor makes sense.

This is up to you. In most cases, it will probably be sufficient to create a destination for each scope or type of communication. You could imagine having a destination for library-type object. You could create my_library_application.interact and on that destination define different Request handlers for searching, retrieving and submitting information. You could fit the entire library interaction API onto that one destination, and retain a pretty orderly internal application structure.

Later, you may want to add functionality to your application for libraries to synchronise with each other, and this would probably make sense to segment out into it's own destination, for example my_library_application.sync. That way, other library instances across the network could simply listen for my_library_application.sync announces on the network, and set up peering with them once they are heard (and therefore discovered). The entire API for handling the sync could then happen as Request/Response handling on the my_library_application.sync destination. Something like this is how LXMF Propagation Nodes discover and sync with each other.

I'm thinking my app needs 3 RNS destinations:
1 To announce availability of a new media item for publication (to anyone / the world)
An RNS annoounce is perfect for this, with the appData being an IPFS hash. Not sure tho, as it looks like an announce is sent when the app is started automatically.
2 Receiving an acknowledgement from a publisher they have pinned the media in IPFS
3 Receiving a message that publishing to a specific platform has completed

Running the actual app communication and state syncronisation over Destination announces is very bad application design, since you are basically flooding the entire network with your app data.

Also, the network-maintaining announce-rate limiting will most likely quickly kick in and put your announced destinations on pause (plus invoke a delay penalty). This means that your application will stop working as intended. What the maximum announce rate you can run is, will vary from network segment to network segment, and is something that is in the control of each network segment owner themselves.

Much better to use the announce mechanism to discover peers, and then periodically have them communicate with each other about the state of various media items. For a very efficient way to converge a state-database over a heterogenous network topology, feel free to scissor what you can use from LXMF. It is super efficient, and even very resilient to nodes dropping in and out of the network, while still getting everyone up to date when they become available.

[ThomasFreedman]

I'm thinking my app needs 3 RNS destinations:
1 To announce availability of a new media item for publication (to anyone / the world)
An RNS annoounce is perfect for this, with the appData being an IPFS hash. Not sure tho, as it looks like an announce is sent when the app is started automatically.
2 Receiving an acknowledgement from a publisher they have pinned the media in IPFS
3 Receiving a message that publishing to a specific platform has completed

Running the actual app communication and state syncronisation over Destination announces is very bad application design, since you are basically flooding the entire network with your app data.

I did consider that, but the reason for the broadcast approach is simplicity. It provides the greatest audience scope, and the publisher doesn't need to know anything about recipients in advance. It provides the greatest degree of decentralized publication. Managing application state is very simple for the publisher.

Also, the amount of app data in these announcements is very small (46 bytes), and it's unlikely a content generator will create very many new publications rapidly. However, that isn't neessarily true for a content aggregator class of publisher.

I appreciate the info about overwhelming the announce system though, that's something I didn't consider but would need to plan for to limit announcement rate if I use this global broadcast approach, and you've given me good reasons to look at other approaches.

The initial primary use case is for multiple publishers to utilize a content distribution service, so it might be better to flip things around and have the content distributor(s) advertize their existence with the periodic announcements like NN nodes use, and have publishers respond to those by establishing a request / response comm link, over which a list of media files and their acknowledgements could be communicated. But that doesn't address individual users as content consumers.

It's also important for my app to accomodate some content producers that will not be online continually. They may only be online long enough to transfer their content to an always online content distributor.

I anticipate some publishers may want more of a typical publisher / subscriber model, but I wanted to accomodate a wide listening audience (i.e. individuals, or search engines that monitor such announcements to build IPFS search databases) for this initial release.

The typical publisher / subscriber model is a centralized, old school concept, and I don't think it wise to put much effort (if any) into such infrastructure. Rather, providing a means of content distribution with the least resistance to the greatest number of people is the best way to address censorship.

There may be a tradeoff between censorship resistance and revenue generation. I feel that those who focus on the later by using the subscriber content for a fee model are limiting their audience and exposing themselves to coersion, control and censorship.

Much better to use the announce mechanism to discover peers, and then periodically have them communicate with each other about the state of various media items.

I don't see this as a pure peer to peer type of problem my application is being built to solve, rather I see it as an application to help content producers protect their content using IPFS. What sharing peers do with each other is out of scope. Getting media content into IPFS and into the hands of content distributors is primary. However, I believe publishers will want to promote using IPFS nodes such as my Pirate Box & Pirate Stick to obtain their content directly from IPFS, and be informed of the existence of that content via RNS.

For a very efficient way to converge a state-database over a heterogenous network topology, feel free to scissor what you can use from LXMF. It is super efficient, and even very resilient to nodes dropping in and out of the network, while still getting everyone up to date when they become available.

Thank you, I will look into that. You've certainly given me much to think over here.