Multiple node routing using prefixes

#1

Hello, I can see router linking gets discussed a lot here but it’s usually about performance or HA, so I’m hoping this is a unique discussion as neither are an issue in my situation.

The wider networks that I’m running clients on are often unstable so I’d like to reduce the connections we need to monitor by creating a single level star network of WAMP routers. Many clients would connect to a local node which maintains a single connection with the central node to pass messages back & forth. The routing logic would be very simple, just using an identifier of each local node as a prefix.

I’m currently looking at implementing this by writing a router component to run on each local node which registers prefix matches on both the local and central router. It would then repeat the calls and publish messages on the other node.

Is creating this client to bridge the nodes the best way to do this kind of prefix-routing right now? Is anyone aware of any prior work that may have been done on this problem?

Thanks!

#2

Hi there,

Many clients would connect to a local node which maintains a single connection with the central node to pass messages back & forth.

that’s exactly what rlinks (“router-to-router links”) do. well, plus more.

rlinks form the basis to create “clusters”, which then are just a set of nodes with rlinks in a defined topology, and with consistent authorization rules (permissions in the WAMP URI space).

That orchestration and automatic management is a feature of our commercial version Crossbar.io FX, but the underlying basis at the WAMP transport level (that is rlinks) is OSS.


finally, rlinks are specifically designed to work with WAN topologies, eg where some nodes sit behind NATs or such, and might only be able to do outgoing network connections to their upstream node in eg the cloud, forming a tree topology.

rlinks are quite newish, but I would call it “beta” now (we did a bunch of alpha iterations already), and we’d love to get user feedback and more exposure and testing!

Cheers,
/Tobias

Cheers,
/Tobias

#3

Thanks for the information and helpful links Tobias!

I’ve tried implementing rlinks but I’m not sure which of my issues I’m facing are bugs or expected behaviour. Could you help me out? The only showstopper is my final point where registered methods are not recoverable after an outage.

I’m testing with a two node, one client setup each running in its own docker container:

Central CB <-- Local CB <-- Client AB

I have a “Central” crossbar node, which accepts an rlink from a “Local” crossbar node. Plus an autobahn python “Client” connected to Local

When Local starts up:
1. A number of wamp.* calls are made from Local to Central which can get blocked by custom authorizers. Is there a list of what should be accepted by custom authorizers here?

2. Local tries to re-register Central’s custom authenticators and authorizers on Central (and fails). Why? :confused:

When the Client registers a method via Local:
3. Local appears to register the method twice and fails the second time due to conflict, possibly it’s replicating its own registration back from Central?

When Central stops/dies:
4. Client remains connected to Local and does not see any connection issues, is that expected?

When Central recovers:
5. Local reconnects but does NOT re-register registrations made by Client previously. How should the system recover after an outage of Central?

Crossbar configs for reference:

Central config.json 
{
    "version": 2,
    "controller": {
        "id": "central"
    },
    "workers": [
        {
            "type": "router",

            "options": {
                "pythonpath": [
                    ".."
                ]
            },
            "components": [
                {
                    "type": "class",
                    "classname": "authenticate.Authenticator",
                    "realm": "office",
                    "role": "central"
                },
                {
                    "type": "class",
                    "classname": "authorize.Authorizer",
                    "realm": "office",
                    "role": "central"
                }
            ],
            "realms": [
                {
                    "name": "office",
                    "roles": [
                        {
                            "name": "central",
                            "permissions": [
                                {
                                    "uri": "central.",
                                    "match": "prefix",
                                    "allow": {
                                        "register": true
                                    }
                                }
                            ]
                        },
                        {
                            "name": "local",
                            "authorizer": "central.authorize.local"
                        },
                        {
                            "name": "client",
                            "authorizer": "central.authorize.client"
                        }
                    ]
                }
            ],
            "transports": [
                {
                    "type": "web",
                    "endpoint": {
                        "type": "tcp",
                        "port": 8070
                    },
                    "paths": {
                        "rlink": {
                            "type": "websocket",
                            "auth": {
                                "cryptosign": {
                                    "type": "dynamic",
                                    "authenticator": "central.authenticate.rlink",
                                    "authenticator-realm": "office"
                                }
                            }
                        },
                        "client": {
                            "type": "websocket",
                            "auth": {
                                "anonymous": {
                                    "type": "dynamic",
                                    "authenticator": "central.authenticate.client",
                                    "authenticator-realm": "office"
                                }
                            }
                        }
                    }
                }
            ]
        }
    ]
}
Local config.json 
{
    "version": 2,
    "controller": {
        "id": "local"
    },
    "workers": [
        {
            "type": "router",
            "realms": [
                {
                    "name": "office",
                    "rlinks": [
                        {
                            "id": "rlink_local_central",
                            "authid": "local",
                            "realm": "office",
                            "transport": {
                                "type": "websocket",
                                "endpoint": {
                                    "type": "tcp",
                                    "host": "central",
                                    "port": 80
                                },
                                "url": "ws://central/rlink"
                            }
                        }
                    ],
                    "roles": [
                        {
                            "name": "client",
                            "authorizer": "central.authorize.client"
                        }
                    ]
                }
            ],
            "transports": [
                {
                    "type": "websocket",
                    "endpoint": {
                        "type": "unix",
                        "path": "/host/socket"
                    },
                    "auth": {
                        "anonymous": {
                            "type": "static",
                            "role": "client"
                        }
                    }
                }
            ]
        }
    ]
}