REST Subscriber Support for Dynamic Wildcard/Regex Topics (x/post from Github)

It’s currently very difficult (almost impossible) to subscribe to topics a client or broker app might create. For example, let’s say i want each user to have their own channel, maybe tracking their e-com orders, or location.

The topic might be com.app.users.{id} or channels.orders.order-{id}

The REST publisher can obviously create whatever topic it likes, but the REST subscriber must specify a static topic identifier. That’s great for public channels, or perma-channels, but if the topic is created on the fly, how can the REST subscriber know about it?

If a topic must be specified, would it be possible to use wildcards or regular expressions? To use something like the config below?

{
    "type": "class",
    "classname": "crossbar.bridge.rest.MessageForwarder",
    "realm": "realm1",
    "extra": {
        "subscriptions": [
            {
                "url": "https://subscriber.app/orders",
                "topic": "com.myapp.orders.*"
            },
            {
                "url": "https://subscriber.app/any",
                "topic": "\b[a-z]."
            },
            {
                "url": "https://subscriber.app/users",
                "topic": "^users."
            }
        ],
        "method": "POST",
        "expectedcode": 200,
        "debug": true
    },
    "transport": {
        "type": "websocket",
        "endpoint": {
            "type": "tcp",
            "host": "127.0.0.1",
            "port": 8080
        },
        "url": "ws://127.0.0.1:8080/ws"
    }
}

prefix and wildcard subscriptions can be setup like this:

{
   "type": "class",
   "classname": "crossbar.bridge.rest.MessageForwarder",
   "realm": "realm1",
   "extra": {
      "subscriptions": [
            {
               "url": "https://httpbin.org/post",
               "topic": "com.myapp.",
               "match": "prefix"
            }
      ],
      "method": "POST",
      "expectedcode": 200,
      "debug": true
   },
   "transport": {
      "type": "websocket",
      "endpoint": {
            "type": "tcp",
            "host": "127.0.0.1",
            "port": 8080
      },
      "url": "ws://127.0.0.1:8080/ws"
   }
}

this will setup a prefix-subscription. you can also use “wildcard” instead of “prefix”.

eg

{
     "url": "https://httpbin.org/post",
     "topic": "com..topic1",
     "match": "wildcard"
}

Thanks for re-posting this. I got it working successfully but it’s slightly tricky to handle in a REST way. The two issues which make it a little difficult are:

a) The topic_id isn’t included in the payload (which just have args + kwargs).
b) The “matching” part isn’t accessible either.

Here’s how i’d (ideally) love to be able to do it (asterisks replacing dots for readability):

topic: com.myapp.*
url: comain.local/something/{matching_str)

topic. users.*.topic1
url: domain.com/api/users/{matching_str|/topics/topic1

rgd a)+b): yes, correct, not included currently.

we could add that in just a couple of lines in callee.py in a non-intrusive way. it’s just a matter of enabling “call details” when registering the WAMP procs there and forwarding the details to the HTTP endpoint. the WAMP call details contain the concrete URI (that matched a pattern).

if you want to see that, please file an issue first on the crossbar repo, fwiw, I’d support adding this …

rgd the 2nd part:

  • on the WAMP side inside callee.py, we could add the necessary stuff also easily, as autobahn supports decorator based WAMP URIs that auto-parse named and typed components. eg com.myapp.product.<product:int>.update

  • on the HTTP endpoint request side: this needs a bit of new code, because the URI components coming from the WAMP side need to be mapped into the HTTP URL forwarded to. easy as well, just .format() the pattern (domain.com/api/users/{matching_str|/topics/topic1) configured with the components parsed

if you want to see that, please file a separate issue first on the crossbar repo, fwiw, I’d support adding this …

note: those 2 issues are related, but not the same, so I think they should be separate

That’s encouraging! I’ll add them in.