Franca with Common API server - continuation

#1

Hi,

since I cannot reply to original “Franca with Common API server” I doing it here on the separate topic (sorry for that, if I am messing with some rules on this forum).
So basically it is possible to have JS client and C++ service, both generated from Franca IDL (fidl), communicating with each other via WAMP Router. Here are some hints:

  1. start with something simple like:

package commonapi

interface HelloWorld {
version {major 1 minor 0}
method sayHello {
in {
String name
}
out {
String message
}
}
}

  1. Generate JS side, based on AutobahnJS. The problem here is that I could not find standalone generator, so had to setup my Eclipse environment to have Franca option enabled. It cost me one evening (fighting with package dependencies), but at least now I have Eclipse supporting me in Franca IDL (syntax, and stuff). As result I got Proxy generated and Blueprint showing how to use it.
  2. For the Common API C++, that’s the tough part. There are several ingredients needed:
    remark: start with https://github.com/GENIVI/capicxx-dbus-tools/wiki/CommonAPI-C---D-Bus-in-10-minutes for general understanding
  • bunch of libraries, like boost, CommonAPI and CommonAPI-WAMP (autobahncpp and msgpack are “header” libraries) (see remark above)
  • actual service implementation (see remark above)
  • generated Common API C++ service files (see remark above)
  • generated Common API C++ WAMP runtime files (https://github.com/GENIVI/capicxx-wamp-tools)
    Then build everything up, and that’s mostly it. Of course there will be several problems with dependencies, compilation, linking, etc etc, but again, going through CommonAPI (DBUS based) tutorial helps a lot. Running it, at least on Windows 7, did work for the first time. I had to slightly modify CommonAPI WAMP library, otherwise my system was throwing exceptions, especially on mutex usage. Also I could not find a place to configure CAPI to connect to the certain port, so hardcoded it in WAMPConnection.cpp
  1. Setup WAMP Router. Well I guess I don’t have to describe it here :slight_smile: One important thing is to have two endpoints configured, one for JS (websocket) and one for C++ (rawsocket). C++ does not communicate over websockets with the router, I spent entire evening trying to connect to the router from C++service until I found it on some forum (ok, to be honest, I didn’t read Crossbar docs about router configuration, my fault :slight_smile: ).
    So eventually I was able to call sayHello method from JS and get the result (concatenated strings).

CommonAPI WAMP is still missing several features, like franca attributes or fireforget methods, and what’s worse I can’t see any activities there. Last commit was about 2 years ago. Like there is no interest or something, pity.

regards,
Michal

1 Like
#2

Hi Michal,

that is awesome! thanks for sharing your work and experiments!

So basically it is possible to have JS client and C++ service, both generated from Franca IDL (fidl), communicating with each other via WAMP Router

does it work for both patterns supported in WAMP, that is Publish & Subscribe and Remote Procedure Calls?

ah, sorry btw for the slow response … we’ve been presenting on IAA as partner of Continental this week: https://www.linkedin.com/posts/oberstet_automotive-data-markets-activity-6577214409804918784-R7HM

Continental presented an automotive data markets platform based on Crossbar.io (100% WAMP) and our new XBR tech which builds on that (https://xbr.network/)

Anyways, totally busy during the last days, but at least I noticed your post - and asked around couple of guys: France + WAMP is highly interesting=) forwarded infos … are you based in germany?

Cheers,
/Tobias

#3

Hello Tobias,

Well, I’ve tried that only for RPC, but as I can see in CommonAPI WAMP spec Publish/Subscribe is also possible via “broadcast” item in Franca IDL (unfortunately attributes, which were extensively used in my previous projects, are not supported). I will check that soon and let you know.

Anyway, Franca is already quite popular… In automotive industry CommonAPI runs on top of SOME/IP, which in turn is supported by AUTOSAR Classic, and new Adaptive AUTOSAR heavily relies on that (it is the main communication mean). WAMP is a new thing for me, I know Websockets quite well, I’ve been using MQTT over Websockets since quite some time. My latest discovery was CommonAPI over WAMP and simply wanted to play a bit with that. I found the post on your forum, someone was asking about details. I could not find anything useful on the internet so I set it up by myself and just wanted to share some thoughts about it. I think there is a potential in there, meaning CommonAPI/SOMEIP for internal Car infrastructure and CommonAPI/WAMP for “external” communication, however I found (and wrote it in my post) that nobody continues working on CommonAPI over WAMP (the latest commits are 2 years old). And Crossbar might be connecting these two “realms” :slight_smile: (kind of gateway between SOME/IP, Websocket and Rawsocket clients - on the top there would be still one common layer -> CommonAPI).

As for the Crossbar, I’ve found it because of CommonAPI/WAMP research. Now I can see that it might be good solution in my projects, where I usually sort of emulate Common/Control channel over MQTT (of course apart from publishing actual data). I didn’t have much time, tried quickly to install Crossbar on OperWrt environment (MIPS SoC HW platform), but turned out that some packages (NaCL for instance) require manual building, so I have to setup cross-compilation environment first. Once I am done, I will try it out, do some tests and see how it goes. Then I can publish some thoughts.

OK, that’s all for now.

Thanks for contacting me.

Have a good weekend,

Michal Kurek

#4

Hi Michal, this is awesome! small world;) let me come back on the things you raised in detail (but not today … need to catch up some sleep;) just 2 things:

a) yes, I know EB … they are around the corner, and I have a personal friend there you might know: https://www.linkedin.com/in/joergscherer

b) here are some quick preview pix (better pix + videos coming soon) from the IAA … there was the hackathon, and there was the presentation of the data monetization platform on the main stage, based on Crossbar.io and XBR:

cheers,
/Tobias

#5

No, sorry, I don’t know Joerg,

for IAA - was there a digital cockpit presentation by Conti, the one which raises (in Autonomous driving mode) and goes down (in Manual driving mode), Android and QNX based (on top of Hypervisor)? That was IAA Demo for Conti that I was working on.

#6

“sort of”:wink: yeah, emulating X over Y is always possible (talking TCP over postcards whatever) … BUT (IMO): that is not what I want as an app developer … reinvent a half baked emulation layer before I even start with my app. half baked RPC over MQTT, eg as in: you need ephemeral topics for the request reply on an RPC, what about failures and timeout conditions? what about app payload format and serialization? and so on. in any case, RPC, and routed RPC in particular is a strength of WAMP.

https://wamp-proto.org/routing.html
https://wamp-proto.org/comparison.html

rgd websocket: yeah, this was a great success. I’ve got into that 2012 participating in the IETF WG specifying websocket;)


[RPC + PubSub?] … I will check that soon and let you know.

that would be highly interesting for me: can we model everything that WAMP provides (naturally) in Franca?

if so, obviously given the traction Franca has in automotive, supporting WAMP in France to generate code bindings etc would be neat

I am aware of the in-car protocol, and also car-to-cloud and cloud-to-car protocol things you raise … and I am aware of some other aspects (in-car WAMP …) as well - can’t talk here though;)

having said that, the also is Protobuf and Flatbuffers … and use of that for … stuff ^ (again … can’t go into here).

BUT: there is upcoming Flatbuffers support as a “serializer” for WAMP. and this is for both WAMP itself, as well as the app payload (which we already do in XBR). if you wanna have a look, here is code (this is schema for WAMP itself):

and here is an example of a WAMP app level API defined in Flatbuffers:

IOW: we will soon have official WAMP protocol bindings and generators based on Flatbuffers

crossbar also supports that. the interesting part is the ability to strictly (statically typed) define app payloads. performance wise we don’t really need it … here are numbers comparing serializer performance if you are interested:

https://crossbario.com/docs/benchmarks/serialization/index.html

that is, crossbar and autobahn (python) on pypy runs at >500k msg serializations / sec on a single cpu core.


rgd the digital cockpit: yes, that was in a separate room area (not in the pix above) with a complete car cabin built up etc - I didn’t had the time to dive into that or take pix:(

#7

Hi michalkurek sir,

Even am trying the same actually. But am not getting how to write the client side generated helloworls-example.js file. Am completely new to this topic do I get any any suggestion or any referral work so that I can try my work please.kind request please.

#8

Sorry, I was quite busy last week.

that would be highly interesting for me: can we model everything that WAMP provides (naturally) in Franca?

I would phrase it differently: can we model everything that Franca needs in WAMP.
And I think yes, well, at least 95% of Franca specification that I know of.
But there is a problem I already mentioned - Javascript generator, no one works on that anymore, so there is just 50% of Franca supported there. Ok, if you know XText stuff, then you could extend it, at the end “attribute” in Franca is a combination of “methods” and “broadcast” and both are already supported. So matter of days, maybe weeks, not more.

#9

@siddharthav: you should first install Franca as Eclipse plugin, instructions are here: https://github.com/franca/franca/wiki/Franca-Quick-Install-Guide, remember that you need Franca 0.13.0

#10

and most likely you will struggle with package dependencies in Eclipse, in my case I could have either CommonAPI C++ generator working OR Javascript (Autobahn based). For C++ you don’t need it anyway, there is a standalone generator, working fine, tested in many projects.

#11

Hi sir,
Sorry for the delayed response. I have already installed franca in eclipse and all the stuffs working good. I even tested the example 10 and example 77 from https://github.com/GENIVI/capicxx-wamp-tools/tree/master/org.genivi.commonapi.wamp.examples. But my doubt is,

I just wrote the below code:

package commonapi

interface HelloWorld

{

 version { major 0 minor 1 }

 method sayHello

   { 

     in 

         {  String name }

     out 

         { String message }

   }

}

Franca 13 generated the below two files:

HelloworldClientBlueprint.js
HelloworldProxy.js

my doubt is:

  1. Should we use the generated Blueprint file as our .js file to create the client application (or) should we create our own new .js file?

  2. In case we need to use our own new .js file, then what is this (auto) generated HelloworldClientBlueprint.js file here for? How does this auto generated file help us? (Because in D-bus case we need to create the new client and server files and we will be calling the generated files in the handwritten files I worked on that too and that well too. But am really newbee in frontend i,e .js, HTML and all so and also am not getting any referral work to catch up the idea)

  3. As per my understanding, we need to call both - proxy.js and created .js file - in HTML file. as below:

Is my understanding correct? If not please let me know the proper direction.

#12

Hi,

Proxy is the most important part,

Blueprint is just to show you how to use Proxy in order to communicate with a service.

It is up to you to reuse it or implement something from the scratch, depending how it fits to the rest of your project.

Treat Blueprint as example, that’s it.

#13

ok sir thank you so much for your input. Will look into it.

  1. As per my understanding, we need to call both - proxy.js and created .js file - in HTML file. as below:
           <script src="gen/org/example/SimpleUIProxy.js"></script>
              <!-- own JS files -->
          <script src="js/example-application.js"></script>

Is my understanding correct? If not please let me know the proper direction.

#15

As I understand it is working now :slight_smile:
I was going to answer that you probably didn’t include autobahn js library in your project.
In my case it was easy since I was using create-react-app, then “npm install autobahn” and “import * as autobahn from autobahn” and that was it.

1 Like
#16

yes sir, I didn’t add that autobahn.min.js earlier now I got the connection actually. Now the issue is I am not getting how to call the method sayhello in javascript to get the respose from the server. i.e. : in javascript part I have

                /**
               * Async callback in response to a 'sayHello' call.
               * 
               * @param cid the call id
               * @param message
                 */
                      proxy.replySayHello = function(cid, message) {
                          // your code goes here
                        };


          /**
            * API function provided by the proxy for calling method 'sayHello'.
           * @param name
          */
                        proxy.sayHello(name);

This is the first time I am working on js stuff I already lost a week for this. I read many js codes and forums too but literally not able to undrestand how to call that methos in .js. your sugeestion in this issue will really help ma a lot. Kind request sir.

#17

Something like this works for me:

HelloWorldProxy.prototype.connect = function(address) {
	var _this = this;

	_this.connection = new autobahn.Connection({
		url: address,
		realm: 'realm1'}
	);

	_this.connection.onopen = function(session, details) {
	console.log("Connected", details);
        _this.session = session;
        if (typeof(_this.onOpened) === "function") {
			_this.onOpened();
	}

		// subscribing to all broadcasts
	}
	
	_this.connection.onclose = function(reason, details) {
		console.log("Connection closed", reason, details);
		if (typeof(_this.onClosed) === "function") {
			_this.onClosed(reason);
		}
	}
	console.log("Connecting to server...");
	_this.connection.open();
}; 


var proxy = new HelloWorldProxy();
proxy.connect('ws://localhost:8089');

proxy.onOpened = function() {
    proxy.sayHello("michal");
}; 

proxy.replySayHello = function(cid, message) {
    console.log("got answer: " + message)
}

Can you see connection established with CommonAPI C++ service? And evidence that the function on the service side was called?

1 Like
#18

yes sir, Below is my code, Here you can check the connectivity of client with the server. Its connecting for me. But after trying your instruction am getting

Once we start server we will get : “connected to server” message at status label in html.
If we stop the server we will get : " Disconnected " message.

But in console am getting below erroe:
’ TypeError: this.session is null
sayHello file:///home/siddhu/helloworldserver/src/hello/client/js/HelloworldProxy.js:20
file:///home/siddhu/helloworldserver/src/hello/client/hello.html?input_test=:80’

May I know what wrong am doing here. Below is my code. Here am writing the js part in .html file only I tied by importing blueprint even then the same error persists for me.

        <!DOCTYPE html>
       <html>
       <head>
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
     </head>
      <body>
         <h1>Franca and CommonAPI C++ WAMP Example</h1>
      <p>This example uses WAMP communication to interact with a C++     server using CommonAPI.</p>
      <br/><br/><br/><hr/>
      <form>
      <br/><br/><br/>

<label>Result: <output id="message" /></label><br/>
     	<br/><hr/><br/><br/>

<label>Status: <output id="connectionStatus"></output></label>
    </form>


       <script src="js/autobahn.min.js"></script>
       <script src="js/HelloworldProxy.js"></script>
       <!--<script src="js/HelloworldClientBlueprint.js"></script> -->

        <script type="text/javascript">
           console.log("I am alive!");

       
         var proxy = new HelloworldProxy();

           proxy.connect('ws://127.0.0.1:8080/ws');

          proxy.onOpened = function() {
     // your code goes here

        console.log('The connection has been opened!')
        setStatus("Connected to network.");
              proxy.sayHello("michal");
          };

              proxy.onError = function() {
          // your code goes here
              console.log('error has been occurred')
          setStatus("error");
                    };

         proxy.onClosed = function(event) {
         // your code goes here

             console.log('The connection has been closed!')
              setStatus("Disconnected");
           }

               proxy.replySayHello = function(cid, message) {
                //document.getElementById("message").value = "myName";
               //your code goes here
                   console.log("got answer: " + message)
                    };




         
       proxy.sayHello("siddh");

     function setStatus(text) {
        document.getElementById("connectionStatus").value = text;
     }

  </script>
#19

Hi sir,.
On quick workaround I think client is connecting to the node not the server. I am getting “Disconnected” message on closing the node. But by keeping the node open and server being closed I am getting “Connected to server” message only.

There is no function call occuring at the server side from the client.

Below is the crossbario node log after starting:

             2019-10-11T00:28:29+0530 [Router       4527]           Router.attach(session=8828642137631745): attached session 8828642137631745 to router realm "realm1"
           2019-10-11T00:28:29+0530 [Router       4527] Router.attach(session=3850913057074641)
          2019-10-11T00:28:29+0530 [Router       4527] Router.attach(session=3850913057074641): attached session 3850913057074641 to router realm "realm1"
           2019-10-11T00:28:29+0530 [Router       4527] Router.attach(session=7780073192865923)
          2019-10-11T00:28:29+0530 [Router       4527] Router.attach(session=7780073192865923): attached session 7780073192865923 to router realm "realm1"
             2019-10-11T00:28:38+0530 [Router       4527] Router.detach(session=3850913057074641)
            2019-10-11T00:28:38+0530 [Router       4527] Router.detach(session=3850913057074641): detached sessions [3850913057074641] from router realm "realm1"
            2019-10-11T00:28:38+0530 [Router       4527] Router.detach(session=8828642137631745)
          2019-10-11T00:28:38+0530 [Router       4527] Router.detach(session=8828642137631745): detached sessions [8828642137631745] from router realm "realm1"
           2019-10-11T00:28:39+0530 [Router       4527] Router.attach(session=7186179133767566)

But there is no evidence of funcion call at the server side.
Any suggestions on this please?

#20

hi sir,

below is my config.json file is it ok. Why I am pointing this is in your above post you mentiones to configure two endpoints may I am doing anything wrong here please…

          {
"version": 2,
"workers": [
    {
        "type": "router",
        "realms": [
            {
                "name": "realm1",
                "roles": [
                    {
                        "name": "public",
                        "permissions": [
                            {
                                "uri": "",
                                "match": "prefix",
                                "allow": {
                                    "call": true,
                                    "register": true,
                                    "publish": true,
                                    "subscribe": true
                                },
                                "disclose": {
                                    "caller": false,
                                    "publisher": false
                                },
                                "cache": true
                            }
                        ]
                    },
                    {
                        "name": "user",
                        "permissions": [
                            {
                                "uri": "",
                                "match": "prefix",
                                "allow": {
                                    "call": true,
                                    "register": true,
                                    "publish": true,
                                    "subscribe": true
                                },
                                "disclose": {
                                    "caller": false,
                                    "publisher": false
                                },
                                "cache": true
                            }
                        ]
                    }
                ]
            }
        ],
        "transports": [
            {
                "type": "web",
                "endpoint": {
                    "type": "tcp",
                    "port": 8080
                },
                "paths": {
                    "/": {
                        "type": "static",
                        "directory": "../web"
                    },
                    "ws": {
                        "type": "websocket",
                        "auth": {
                            "anonymous": {
                                "type": "static",
                                "role": "public"
                            }
                        }
                    },
                    "call": {
                        "type": "caller",
                        "realm": "realm1",
                        "role": "public",
                        "options": {
                            "debug": true
                        }
                    }
                }
            },
            {
                "type": "rawsocket",
                "endpoint": {
                    "type": "tcp",
                    "port": 8000
                },
                "auth": {
                    "anonymous": {
                        "type": "static",
                        "role": "public"
                    },
                    "wampcra": {
                        "type": "static",
                        "users": {
                            "homer": {
                                "secret": "secret123",
                                "role": "user"
                            }
                        }
                    }
                }
            },
            {
                "type": "rawsocket",
                "endpoint": {
                    "type": "unix",
                    "path": "/tmp/crossbar.sock"
                },
                "auth": {
                    "anonymous": {
                        "type": "static",
                        "role": "public"
                    },
                    "wampcra": {
                        "type": "static",
                        "users": {
                            "homer": {
                                "secret": "secret123",
                                "role": "user"
                            }
                        }
                    }
                }
            }
        ]
    }
]
 }

Any suggestion will help me a lot sir.

#21

this is my config.json (transports section):

"transports": [
                {
                    "type": "web",
                    "endpoint": {
                        "type": "tcp",
                        "port": 8089
                    },
                    "paths": {
                        "/": {
                            "type": "websocket"
                        },
                     "info": {
                                "type": "nodeinfo"
                        },
                        "ws": {
                            "type": "websocket"
                        }
                    }
                },
				{
                    "type": "rawsocket",
					   "serializers": ["json", "msgpack"],
					   "endpoint": {
						  "type": "tcp",
						  "port": 8090
					   }
                }
            ] 

C++ service:

int main() {
    std::shared_ptr<CommonAPI::Runtime> runtime = CommonAPI::Runtime::get();
    std::shared_ptr<HelloWorldStubImpl> myService =
    	std::make_shared<HelloWorldStubImpl>();
    runtime->registerService("local", "commonapi.HelloWorld", myService);
    std::cout << "Successfully Registered Service!" << std::endl;

    while (true) {
        std::cout << "Waiting for calls... (Abort with CTRL+C)" << std::endl;
        std::this_thread::sleep_for(std::chrono::seconds(30));
    }
    return 0;
 }

C++ service stub implementation:

void HelloWorldStubImpl::sayHello(const std::shared_ptr<CommonAPI::ClientId> _client,
	std::string _name, sayHelloReply_t _reply) {
	    std::stringstream messageStream;
	    messageStream << "Hello " << _name << "!";
	    std::cout << "sayHello('" << _name << "'): '" << messageStream.str() << "'\n";

    _reply(messageStream.str());
};

in WampConnection.cpp (capicxx-wamp-runtime package) replace rawsocketEndpoint with this:
static const boost::asio::ip::tcp::endpoint rawsocketEndpoint(boost::asio::ip::address::from_string("127.0.0.1"), 8090);

and javascript part you already know.

1 Like