Spring Cleaning Plan

This is a vague set of proposals, that I would like us to consider going forwards for this library.

Drop WebSocket Implementation

We maintain our own websocket implementation as part of the source. This works fine on Android and have been working for multiple years. Autobahn is mainly a WAMP library, we did our own WebSocket implementation at time when there weren’t many of them out there for Java. In 2020, there are quite a few, stable and mature.

Since we have been using Netty’s websocket implementation for non-android platforms, it might make sense to adopt that on Android as well. Netty does not have any external dependencies, which means we won’t be ballooning the distributed library’s size.

This ultimately reduces any future maintenance burden on the library.

Remove RawSocket POC

I did a proof of concept WAMP over RawSocket handshake implementation a while ago. It didn’t go any further, however the code exists and isn’t really useful at all.

Is there a chance we’ll ever want to do WAMP over RawSocket in Java ?

Revisit Reflection-based roles

That feature was contributed by community in 2017 by that PR https://github.com/crossbario/autobahn-java/pull/356, a few samples were added for it as well https://github.com/crossbario/autobahn-java/pull/380

We haven’t improved that or implemented any of its TODOs, it also has some issues that are mentioned in the original pull request as well. IMO it doesn’t fit with the goal of the library to keep that code lurking around in such an unfinished state.

Maybe we could remove that functionality all together ?

Remove deprecations

There are a few @Deprecated annotations, we can probably remove their underlying code, its been more than a year.

om26er@KNUC:~/scm/crossbario/autobahn-java/autobahn$ grep -r Deprecated
src/main/java/io/crossbar/autobahn/wamp/types/WebSocketOptions.java:@Deprecated
src/main/java/io/crossbar/autobahn/wamp/Session.java:    @Deprecated
src/main/java/io/crossbar/autobahn/wamp/transports/NettyWebSocket.java:    @Deprecated
src/main/java/io/crossbar/autobahn/wamp/transports/NettyWebSocket.java:    @Deprecated
src/main/java/io/crossbar/autobahn/wamp/messages/Hello.java:    @Deprecated

Regarding the reflection based roles feature - this feature might not be well understood for people coming from dynamic typed languages such as JavaScript or Python, but it is expected in strongly typed languages, such as Java. It allows one to write concise code in order to consume external services, and to avoid boilerplate code that does boring conversions. It also prevents mistakes in performing these conversions.

My pull request was supposed to be a first step in a long term plan for reflection support for WAMP. In a more advanced phase, one could use AutobahnJava to the tell the router about the signatures of the declared methods of its contracts, and then the router could expose these to other clients, that could generate code (in other languages!) and consume these PubSub/Rpc services. The generated code would allow one to save time writing code that consumes these services, and would prevent mismatch errors. See also the blog post I wrote here.

Unfortunately, this feature did not receive any love, and after my pull request, it was forgotten. No documentation was written, and the feature wasn’t mentioned in the README. The samples I wrote are the only “documentation” available for this, and I find it hard to believe anyone is actually using it. This is disappointing, as I did spend some time implementing this, and I do think that this feature is super useful (in WampSharp, the analogous features are extensively used).

Furthermore, reflection for WAMP was entirely dropped about two years ago, although I never understood why. My general feeling is that WAMP is no longer main priority and that the team moved to work on XBR.

Elad

Hey @darkl, thanks for your detailed feedback and contributing heavily to the overall WAMP ecosystem.

First I want to make it clear that everything that we build on is based on WAMP. It is the core of everything that we have been working on. While we have been busy with the XBR stuff, I assure you that our commitment to WAMP stays the same. More so we have adding many long standing feature requests to Crossbar OSS as well.

Regarding reflection-based roles, I believe it has great potential in terms of functionality, however in the short-term at least it doesn’t look like we’ll be able to extend that, due to resource constraints. However, you are right, we definitely need to add documentation of that, add a few more examples and add hints for it to the main REAMDE file.

Moreover my assertion in my initial post that ‘it doesn’t fit with the goal of the library’ was wrong, as I was told by Tobias as well. The statement was vague clearly.

I look forward to working with you to improving that functionality in the library.

In my conversation with @oberstet, it’s clear that we need to maintain and even improve our WebSocket implementation. One fact I came to know is that it was the first Java implementation of WebSocket.

Regarding, WAMP over RawSocket, it has potential use cases in the server space, especially due to performance reasons, something I wasn’t aware, that Tobias told me about. So we’ll be keeping it and I hope at one stage we’ll be able to “finish” it as well.

Hi Elad,

My pull request was supposed to be a first step in a long term plan for reflection support for WAMP.

I would definitely keep your contribution in AutobahnJava for the time being and take up work on in it again.

One thing I’m not sure about is this: is what you refer to as “reflection” meant to be

a) a Java-only thing, as in reflect a Java native, strongly typed procedure or event handler, so that serialization/deserialization to WAMP dynamically typed payloads (as it is today) works automatically and is type checked strongly automatically?

or

b) is it supposed to work as a strongly-typed application payload format and machinery for WAMP in general, working with (in principle) any WAMP supported language?

If it is about the former, that is cool and there is no hesitation on my side! If it is about the latter, then this is a bigger discussion=)

But first, I fully agree with what you seem to hint at in the blog post:

  • " it might be challenging to consume services which have a large domain"
  • “enjoy advantages of static-typed languages (such as intellisense)”

So WAMP reflection and strongly typed app payload should improve “programming at large” with WAMP (eg many developers, many microservices, etc).

My main thoughts here rgd b) are:

  • I’m not convinced using Java as a (general) DSL for specifying strongly typed application payloads for a general machinery (working cross language) - if that is the idea anyways (which I’m not sure is what you propose … so)
  • for one, I hate Java;) the type system is incomplete (nothing like ML family in that regard), the syntax sucks big time, etc. but that that’s not my main point, but:
  • I think what is needed is a DSL independent of specific WAMP service implementation languages for describing strongly typed WAMP app payloads
  • (run-time) reflection on endpoints (procedures/event handlers) described by that DSL is then just a 2nd step

longer ago, I looked at various options and settled on https://google.github.io/flatbuffers/

since then I contributed to the compiler https://github.com/google/flatbuffers and used flatbuffers quite extensively, not really yet for this reflection stuff, but for all persistency in Crossbar.io. this work (a sidetrack to this issue, but flatbuffer based also) is


the same tech is also used for XBR market maker persistency, eg here are some tables: https://xbr.network/docs/network/reference.html#database-schema

the “table docs” are just generated via sphinx from python classes generated from the flatbuffer schema files using flatc.

I am quite happy with the DB use of Flatbuffers. it is also fast: read side is ~5 million records/sec scan rate in a single threaded PyPy program. Eg we’re using it directly from within Jupyter.


ok, this post is getting too long again;)

long story short: the plan is to build upon flatbuffers and use it as a DSL for describing WAMP endpoints like so (sorry, this is all sketch, and a bit of a mess … but you should get the idea):

https://github.com/crossbario/xbr-app/blob/master/schema/xbr-location-api.fbs


and generating strongly typed Java classes could be done easily, without touching flatc at all:

  • Flatbuffers has a (complete) meta schema https://github.com/google/flatbuffers/blob/master/reflection/reflection.fbs
  • using that, you can represent any flatbuffer schema as a value serialized according to above meta-reflection schema
  • that is, use flatc to compile your FB schema S to Val, and then use whatever language L1 (eg Java or Python or whatever) to read Val and go over iter to generate code in lang L2 (eg Java)
  • this could also be added to Crossbar.io: read schema S into Val, expose the schema info via WAMP - and that means run-time WAMP level reflection comes for free!

Unfortunately, this feature did not receive any love, and after my pull request, it was forgotten. No documentation was written, and the feature wasn’t mentioned in the README.

sorry! :frowning: if this doesn’t collide with ^ (eg because it is actually about a), not b)), we should definitely add README, docs, examples and promote it @om26er

Furthermore, reflection for WAMP was entirely dropped about two years ago, although I never understood why. My general feeling is that WAMP is no longer main priority and that the team moved to work on XBR.

that might have been a mistake:( ready to reopen this … but I think we should really first try to agree on what “reflection” is supposed to mean for everybody;)

XBR: in a way this is true in that I spend many hours on this new tech. however, I now see how XBR can be a natural extension and in fact final conclusion of what all this WAMP effort started! plus, some important business aspects changed (well, didn’t work out as originally planned), and I’m ready to open up lots more rather than dumping it.

anyways, I hope all this isn’t too confusing … meta meta bla bla;)

I’d love to hear your thoughts!

Cheers,
/Tobias

Hi Tobias,

My current concerns (regarding no love for the feature and no documentation, etc) are regarding your a) item.

In the long term, I would like to see a reflection feature for WAMP, which is similar to what you described in your reply. This could be consumed nicely using the reflection based roles feature using code generation. It might also be possible to generate the metadata of the method/topic in some cases based on Java/C#/statically typed language code, for example from a method of the form

@WampProcedure("com.example.add2")
int add2(int x, int y);

We can infer using reflection that this method has a WampProcedure attribute with value com.example.add2, it receives two parameters which are integers, and that it returns an integer. Then we can define this at the router as a procedure named com.example.add2, receiving two positional parameters and returning a single positional parameter, all of type integer.

I agree that this might not work for all cases, and a less automatic api level is also necessary, where we allow the user to define themselves the topics/procedures. Anyway, I’d be happy to see reflection in WAMP and then see where can we take the implementations.

For now, I am worried about the status of “a)” in your message.

Hi Elad,

My current concerns (regarding no love for the feature and no documentation, etc) are regarding your a) item.

thanks for clarification! I definitely want to keep your work … and actually expand on it, making it fully documented, with examples, and essentially bring it to the same level as it is in wampsharp.

we should probably make it visible enough so that we can call-for-help, as we (now me personally, and Omer, both working for Crossbar.io the company) have just so many things on our plate, including stuff that has direct commercial impact.

@om26er so in summary, my view on the 4 areas you raised

  • Drop WebSocket Implementation => definitely not
  • Remove RawSocket POC => we would want to have it eventually, but not a main prio now. keep it.
  • Revisit Reflection-based roles => revisit yes, but not remove. keep it.
  • Remove deprecations => that’s cool of course!

Cheers,
/Tobias