Crossbar 'Studio'

#1

Hi,

Can I ask if anyone is working on a Studio or IDE type application for Crossbar?

0 Likes

#2

What exactly would it do?

···

On Sunday, 3 July 2016 21:22:04 UTC+2, Gareth Bult wrote:

Hi,

Can I ask if anyone is working on a Studio or IDE type application for Crossbar?

0 Likes

#3

Ok, well, an “applications” programmer needs to know about business logic, and to a lesser extent pub/sub and rpc. There is a lot to know about the internals of Crossbar (and autobahn) that could put it beyond the reach of many “normal” applications programmers. So my “vision” would be to have an application that wraps up all the functionality of Crossbar, deferred objects, custom authentication, crossbar configuration etc, into a visual application, so all the programmer needs to do is write pub/sub/rpc code stubs, and the rest is taken care of.

I have done some work on this, my latest Crossbar application is partially “generated” from code stubs, and I have a little GUI running on top of those code stubs which looks quite promising. I thought before going further I would ask if anyone else is working on something similar (or better), or whether this might be of use to others …

In this particular instance I’ve started to store all the code etc in a “mongodb” database, and I’m using “codemirror” as an editor … which leaves some interesting options for code deployment and replication …

···

On Monday, 4 July 2016 05:43:09 UTC+1, Adam Jorgensen wrote:

What exactly would it do?

On Sunday, 3 July 2016 21:22:04 UTC+2, Gareth Bult wrote:

Hi,

Can I ask if anyone is working on a Studio or IDE type application for Crossbar?

0 Likes

#4

I’ve got to be honest, I really don’t like this idea.

If you can’t grok async development then you probably shouldn’t be playing with Crossbar (or Node or anything else of this sort) and hence an IDE that abstracts away some of the important stuff that is happening is, in my eyes, a bad idea.

The road to hell is, as they say, paved with good intentions and one of these good intentions is, in my opinion, attempting to abstract away important concepts.

With that said, I wish you all the luck with your project. I’ll probably just stick to my current tooling though :slight_smile:

···

On Monday, 4 July 2016 11:07:00 UTC+2, Gareth Bult wrote:

Ok, well, an “applications” programmer needs to know about business logic, and to a lesser extent pub/sub and rpc. There is a lot to know about the internals of Crossbar (and autobahn) that could put it beyond the reach of many “normal” applications programmers. So my “vision” would be to have an application that wraps up all the functionality of Crossbar, deferred objects, custom authentication, crossbar configuration etc, into a visual application, so all the programmer needs to do is write pub/sub/rpc code stubs, and the rest is taken care of.

I have done some work on this, my latest Crossbar application is partially “generated” from code stubs, and I have a little GUI running on top of those code stubs which looks quite promising. I thought before going further I would ask if anyone else is working on something similar (or better), or whether this might be of use to others …

In this particular instance I’ve started to store all the code etc in a “mongodb” database, and I’m using “codemirror” as an editor … which leaves some interesting options for code deployment and replication …

On Monday, 4 July 2016 05:43:09 UTC+1, Adam Jorgensen wrote:

What exactly would it do?

On Sunday, 3 July 2016 21:22:04 UTC+2, Gareth Bult wrote:

Hi,

Can I ask if anyone is working on a Studio or IDE type application for Crossbar?

0 Likes

#5

Ok, well … I did it anyway … :slight_smile: … I’ve added a decent terminal emulator and build/run option to encompass Crossbar and any associated “Microservices”, so thinking back to the days of Turbo Pascal, I now have a pretty much working IDE for writing Crossbar based applications … lots of polish left to add, but the next job is to import the remaining parts of itself, into itself, so it essentially becomes a self compiling compiler … :slight_smile: … if the idea appeals to anyone, let me know …

0 Likes

#6

Hi Gareth,

looks interesting!

Maybe you could write a blog entry in which you explain how you would go about building one of the examples using your IDE.

Regards,

Roger

···

Op zaterdag 20 augustus 2016 16:49:26 UTC+2 schreef Gareth Bult:

Ok, well … I did it anyway … :slight_smile: … I’ve added a decent terminal emulator and build/run option to encompass Crossbar and any associated “Microservices”, so thinking back to the days of Turbo Pascal, I now have a pretty much working IDE for writing Crossbar based applications … lots of polish left to add, but the next job is to import the remaining parts of itself, into itself, so it essentially becomes a self compiling compiler … :slight_smile: … if the idea appeals to anyone, let me know …

0 Likes

#7

Yup, will see what I can do … :slight_smile:

0 Likes

#8

Ok, well that was interesting, I seem to have generated some bug fixing work, but it’s all good … :slight_smile:
Tell me what you think fo this;

https://gareth.bult.co.uk/2016/08/23/for-example/

0 Likes

#9

Ok, if anyone’s interested this has now moved on. Probably not suitable for anyone already involved in large projects as it’s a different way of working, but for anyone looking to “play” with Crossbar/Autobahn who’s finding the setup / configuration a little daunting, this might be a good introduction. For what it’s worth, it works to the extent that the IDE is implemented as a project inside the IDE, and runs inside itself. (self compiling compiler, eating own dog food etc etc) … Anyway;

https://gareth.bult.co.uk/2016/11/03/countdown/

Example operation, in the IDE;

  • Project -> Import -> iFlexStudio.proj
  • iFlexStudio -> Run

You now have “itself” (i.e. a running crossbar.io, iFlexCode (microservice) and static content) running inside the IDE and available on on port 8443.
It’s very alpha / bleeding edge, but it’s enough for me to use “it” to develop “it” without any other editors or tools …
I’m looking for one or two people with a little time on their hand who might be prepared to do a little testing / provide some feedback …

0 Likes

#10

Hi Gareth,

sorry for late response (catching up): nice, nice!

It’s astonishing what can be achieved today in terms of UI with modern Web technologies (which I guess this is based on).

Here is my perspective: this won’t be for everyone (in a way, I agree with Adam: if you don’t know about async, you will get stuck no matter what tooling you put on top), but the idea of a Turbo Pascal / Delphi / VBA like IDE for Crossbar.io is nevertheless intriguing. In particular, concepts like “code behind” or getting started quickly by using app templates / scaffolding …

So do you envision this to be more like VBA (with built in Forms capabilities and the like), but using Python/Autobahn instead of VB, or more like Visual Studio, with scaffolding capabilities and multi-language support?

Cheers,
/Tobias

PS: There is another “low barrier” IDE that I find interesting: https://nodered.org/ - I know about a company using Crossbar.io that have built NodeRED “nodes” for WAMP publisher/subscribers/caller/callee … they haven’t released it OSS though …

···

Am Donnerstag, 3. November 2016 19:07:09 UTC+1 schrieb Gareth Bult:

Ok, if anyone’s interested this has now moved on. Probably not suitable for anyone already involved in large projects as it’s a different way of working, but for anyone looking to “play” with Crossbar/Autobahn who’s finding the setup / configuration a little daunting, this might be a good introduction. For what it’s worth, it works to the extent that the IDE is implemented as a project inside the IDE, and runs inside itself. (self compiling compiler, eating own dog food etc etc) … Anyway;

https://gareth.bult.co.uk/2016/11/03/countdown/

Example operation, in the IDE;

  • Project -> Import -> iFlexStudio.proj
  • iFlexStudio -> Run

You now have “itself” (i.e. a running crossbar.io, iFlexCode (microservice) and static content) running inside the IDE and available on on port 8443.
It’s very alpha / bleeding edge, but it’s enough for me to use “it” to develop “it” without any other editors or tools …
I’m looking for one or two people with a little time on their hand who might be prepared to do a little testing / provide some feedback …

0 Likes

#11

Hi, no problem :slight_smile:

Whereas I can’t completely disagree with you in terms of the async stuff, it is surprising how much of it can be removed from the programmer’s domain, being able to use “return” for example in P3 rather than having to insert “returnValue” everywhere does make for more readable code. (I’m still playing with other things to reduce the issue of missed “yield” or inline callback declarations)

I think what I’ve done isn’t something one might naturally ‘envisage’, I’m not trying to specify the UI (at all), what I’m trying to do is to simplify the framework, bear in mind I’m coming at this from a DevOPS + application background where system programming is a necessary evil that most coders will avoid at all costs if they can.

So the idea is that you can select File->New Project from the menu and immediately you have a working server which provides;

  • a fully working crossbar instance

  • http server with a https redirect

  • https server (soon to be http2)

  • a working set of auto-generated certificates

  • a working WSGI mapping to serve up static content (using Falcon, not Flask :slight_smile: )
    File->New Microservice then adds;

  • A stand alone python twistd application instance

  • Automatic connection to crossbar using http client certs (all auto generated)

  • A framework where you can write stub routines without worrying about the assembly

    • so for example, create a folder called app, then a sub folder called example
    • create an item called “test” and set the type to be “python”
    • inside test create a “def function(…)”
    • at “run” time this will automatically make “app.example.test” a registered RPC call
    • application definition, registration, subscription, all are implicit to the coder

If you decide you want a dhparam on your SSL, upload one and the config sees it and adjusts config.json accordingly.

Change config.ini, the system sees it, regenerates config.json and restarts crossbar.

Change your function stub, the system sees it, regenerates the microservice code and restarts the microservice.

… you never “see” config.json, it all comes from a .ini file, so for crossbar we have;

[global]
realm = crossbar
access_log = false
database =
debug = true
protocol = wss
google_id =

[autobahn]
initial_retry_delay = 1.0
max_retry_delay = 10.0
retry_delay_growth = 2.0
max_retries = 999999

[options]
enable_webstatus = false
open_handshake_timeout = 5000
close_handshake_timeout = 5000
auto_ping_interval = 20000
auto_ping_timeout = 20000
auto_ping_size = 32
fail_by_drop = true
max_message_size = 9999999

[role_server]
authorizer = xbr.security.authorize

will exist as authorize in crossbar_io/rpc/xbr/security folder

[role_client]
authorizer = xbr.security.authorize

[transport_server]
port = 8444
ssl = true
key = key.pem
cert = cert.pem
dhparam = dhparam.pem
auth_tls = xbr.security.auth_tls

will exist as auth_tls in crossbar_io/rpc/xbr/security folder

[transport_client]
port = 8445
ssl = true
key = key.pem
cert = cert.pem
dhparam = dhparam.pem
auth_ticket = xbr.security.auth_ticket

will exist as auth_ticket in crossbar_io/rpc/xbr/security folder

[ssl]

for SSL certificate generation

country = UK
state = South Wales
city = Cardiff
company = iFlexRTS Ltd
organisation = iFlexRTS Ltd
hostname = darkstar.iflexrts.uk

``

Then on the microservice side, we have Jinja templating, so we can make a dependent .INI file like this where the stuff in double curlies are pulled in from the crossbar .INI file;

[global]
debug = {{ global_debug }}
realm = {{ global_realm }}
database = {{ global_database }}
user = server
port = {{ transport_server_port }}
ssl = {{ transport_server_ssl }}
cakey = {{ transport_server_key }}
cacrt = {{ transport_server_cert }}
host = localhost
logfile = app

``

Again, you don’t get to see or worry about the microservice startup code, it’s all done for you, ultimately it presents a bit of python that you can start with;

twistd -y

``

Where all you’ve had to do is to write the actual stub (or application) code - you don’t need to know anything about twisted, servers, certificates, etc etc, and when you’re developing, you don’t need to know about twistd, the IDE manages all that for you.

The UI itself can handle an unspecified number of Microservices per project (with one xbar per project), such that when you edit code, ini files etc etc, and then press F8 (Run All), it will work out any dependent changes, re-assemble any framework changes that have resulted, then restart any crossbar or Microservices instances that have been subject to the changes. (each xbar or ms instance runs in a terminal window thanks to xterm.js)

At the same time, the UI can handle an unspecified number of open projects at any one time, so if you need to work on something that does involve a number of crossbar instances, you can load/edit/run then all inside a single IDE instance at the same time.

… which brings me to deployment. The code I’ve just finished (yesterday) converts everything to containers, so when you load a project, it creates a container for that project, and all generated code is sync’d to the container, and when you run the project, it runs in that container. (this is all transparent to the coder) So, when you’ve finished coding, your run-time is actually sitting in a container that you can literally just ship. The IDE itself is automatically deployed with Ansible, so a single command line will automatically create a DigitalOcean instance and deploy, what I hope to add to the IDE is this functionality, so when an application is complete you should be able to select “Deploy to DO” and have it send the container up to an existing (or new) DO instance, install the appropriate firewall/proxy code, install the appropriate DNS A record, and make it live with one click.

What you write “inside” the IDE, whether it’s Bootstrap or ExtJS or DoJo or something else … that would be up to the coder … :slight_smile: … in the case of the IDE, I’m using a combination of OS Javascript and some JS widgets I’ve written myself. Whereas “my” interest is “single page applications”, there’s absolutely no reason why this wouldn’t work just as well for IoT stuff with no UI to speak of.

What I’m going to find really interesting is to be able to add other languages, for example I want to add a “C” assembly so you have the chance to write a “C” based micro-service (again, coder just writes the stubs) in which case you could have a project containing a crossbar, python microservice and a C microservice, all subject to an edit + F8 sequence … real polyglot … :slight_smile: … the ability to call “C” routines directly from Javascript, I dunno, it has some sort of inexplicable appeal … :wink:

0 Likes

#12

I think we can all agree that the peculiar things Twisted does to allow async programming are, well, a little twisted :wink: Thankfully the async/await stuff in Python 3.5 looks to help with that a great deal (The same syntax has been a huge improvement for me in JavaScript :slight_smile:

···

On Friday, 11 November 2016 12:20:02 UTC+2, Gareth Bult wrote:

Whereas I can’t completely disagree with you in terms of the async stuff, it is surprising how much of it can be removed from the programmer’s domain, being able to use “return” for example in P3 rather than having to insert “returnValue” everywhere does make for more readable code. (I’m still playing with other things to reduce the issue of missed “yield” or inline callback declarations)

0 Likes

#13

Absolutely, inadvertently pasting in a txmongo function with a yield and forgetting to add an inlineCallbacks is the bain of my life. However (!) one of the benefits of preprocessing code is that you can change it before the Python interpreter gets it’s hands on it … I’m quite hopeful that with a little help from a decorator and a little more IDE intelligence I’ll be able to remove the need for both inlineCallback declarations and the use of “yield”. All the “system” stuff I used to have to worry about is now all gone (SSL certs, tls authentication, ticket authentication, dynamic auth, config.json, register, subscribe etc etc), my target is to try to remove “all” the things that potentially suck programmer time above and beyond algorithms and syntax.

0 Likes

#14

Ok, got a little distracted by another project, but in a way that’s a good thing because it’s giving me a chance to use what I have so far.

So, may not be everyone’s cup of tea, but if you want all the fun without all the graft, it might be of interest. :slight_smile:

https://gareth.bult.co.uk/2016/12/30/iflexrtsstudio-walkthru/

0 Likes