here is more concrete ask:
1. scale-out: app logic containers need to be able to run collapsed
(same host as router) as well as in separate hosts. Can crossbar.io
manage workers on different hosts?
A Crossbar.io instance can only start workers locally.
This allows you to scale up your app logic (that is run the logic on multiple core on the local machine) by starting more workers managed from Crossbar.io
However, a Crossbar.io instance can (obviously) accept incoming WAMP connections from app components from anywhere. This is what allows scale-out your app logic today - run the components on other hosts.
Those connecting clients won't be managed/monitored by Crossbar.io as workers then though.
Here is where we want to go:
Have a cluster/federation of Crossbar.io instances where you can start workers transparently on any of the nodes. Or have Crossbar.io automatically make placement decisions (like fire up a worker on a least loaded host). Or have Crossbar.io fire up a worker in a OS container (think Docker). Etc.
This latter stuff is pointing to an exciting perspective: Crossbar.io as a complete microsservice platform.
Would be interesting to me where you actually want to go with Crossbar.io in your app/project ...
2. scale-up: need multiple instances for each app container, for
availability as well as throughput scaling.
* How do we do this with RPC registration? the router will not
allow redundant endpoint registration, does it?
Again, in the pipeline: the WAMP "Advanced Profile" talks about this under the term "distributed/partitioned RPC/PubSub".
Regarding HA-Callees: https://github.com/tavendo/WAMP/issues/89
* How do we do this with subscription that only 1 process within
this "redundant processes group" get to process the given message.
Not sure what you mean here .. can you expand on the behavior you envision?
* Can crossbar.io hold connection with the container through a
container-side proxy (i.e. a load balancer fronting the
containers)? Or maybe crossbar.io has a concept of "worker
group" that only 1 instance of such workers will be selected to
process the incoming message ?
Kind of the latter. Crossbar.io will implement different policies, like round-robin, random, .. for directing e.g. a specific call to respective endpoints (callees). There is no LB involved. Crossbar.io is a WAMP-level LB.
From a practical point of view, here is what you can do today for HA:
- Have a hot-standby Crossbar.io instance (that is one already
running, but with _no_ clients connected).
- When the primary fails, failover to the former (either using a LB,
or by taking over the IP of the former)
- When the former fails, all clients (both frontend and backend
components) will loose their connections
- All clients will (should) automatically reconnect (as e.g.
AutobahnJS does), and hence connect to the standby
For short term, this is acceptable, provided it didn't happen often
FWIW, I haven't seen our public Crossbar.io instance (which runs on EC2) collapse even once. It's restarted after Crossbar.io or OS upgrades. That's it.