RPC, problems with the response

#1

Hello everyone,
I use Crossbar.io together with Autobahn|Python Twisted on my Raspberry Pi 2 (Ubuntu Mate).

For some testing i query a sqlite/spatialite-db from an angularJS-App. For the communication i use angular-wamp.

So I have a procedure in Python which do all the heavy work (prepare the query, execute it and produce a geojson-string from the result).

My idea behind this is to have something similar like a vector-tile query based on slippy-map Tiles (zoom/xtile/ytile).

Everything works fine, dispite of one thing. If i execute the call 2-3 or more times i get my response after all calls are finished.

On my Pi 2 i did some console.log() to see what happened inside of the procedure. The procedure works normal and execute one by one, but on the client side i get every result after every call is processed at once.

My question is, when my procedure execute one by one, why did i get the response only after no more execution are going on?

Stripped down python code:

class AppSessionSpatialite(ApplicationSession):

@inlineCallbacks

def onJoin(self, details):

def get_data_db(queryData):

return result

yield self.register(get_data_db, ‘de.map.data.request’)

0 Likes

#2

I think you have to make get_data_db as a coroutine as well, and also your db connection may be blocking the request. Are you using an async driver for the db?

···

On Wed, Aug 26, 2015 at 12:30 PM, Robert Gregat rgr...@gmail.com wrote:

Hello everyone,
I use Crossbar.io together with Autobahn|Python Twisted on my Raspberry Pi 2 (Ubuntu Mate).

For some testing i query a sqlite/spatialite-db from an angularJS-App. For the communication i use angular-wamp.

So I have a procedure in Python which do all the heavy work (prepare the query, execute it and produce a geojson-string from the result).

My idea behind this is to have something similar like a vector-tile query based on slippy-map Tiles (zoom/xtile/ytile).

Everything works fine, dispite of one thing. If i execute the call 2-3 or more times i get my response after all calls are finished.

On my Pi 2 i did some console.log() to see what happened inside of the procedure. The procedure works normal and execute one by one, but on the client side i get every result after every call is processed at once.

My question is, when my procedure execute one by one, why did i get the response only after no more execution are going on?

Stripped down python code:

class AppSessionSpatialite(ApplicationSession):

@inlineCallbacks

def onJoin(self, details):

def get_data_db(queryData):

return result

yield self.register(get_data_db, ‘de.map.data.request’)

You received this message because you are subscribed to the Google Groups “Crossbar” group.

To unsubscribe from this group and stop receiving emails from it, send an email to crossbario+...@googlegroups.com.

To post to this group, send email to cross...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/crossbario/b89d8d7c-ff1d-4612-a1ad-d07c4a794985%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Johnny W. dos Santos

0 Likes

#3

We need to see what you do in the get_data_db procedure. You must not block the reactor … eg need to use appropriate db code as db drivers are blocking most often

···

Sent from Mobile (Google Nexus 5)

Am 26.08.2015 17:30 schrieb “Robert Gregat” rgr...@gmail.com:

Hello everyone,
I use Crossbar.io together with Autobahn|Python Twisted on my Raspberry Pi 2 (Ubuntu Mate).

For some testing i query a sqlite/spatialite-db from an angularJS-App. For the communication i use angular-wamp.

So I have a procedure in Python which do all the heavy work (prepare the query, execute it and produce a geojson-string from the result).

My idea behind this is to have something similar like a vector-tile query based on slippy-map Tiles (zoom/xtile/ytile).

Everything works fine, dispite of one thing. If i execute the call 2-3 or more times i get my response after all calls are finished.

On my Pi 2 i did some console.log() to see what happened inside of the procedure. The procedure works normal and execute one by one, but on the client side i get every result after every call is processed at once.

My question is, when my procedure execute one by one, why did i get the response only after no more execution are going on?

Stripped down python code:

class AppSessionSpatialite(ApplicationSession):

@inlineCallbacks

def onJoin(self, details):

def get_data_db(queryData):

return result

yield self.register(get_data_db, ‘de.map.data.request’)

You received this message because you are subscribed to the Google Groups “Crossbar” group.

To unsubscribe from this group and stop receiving emails from it, send an email to crossbario+...@googlegroups.com.

To post to this group, send email to cross...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/crossbario/b89d8d7c-ff1d-4612-a1ad-d07c4a794985%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

0 Likes

#4

That could be one problem, yes.
I’m using pysqlite.

To open and close the spatiaLite-db i have two separate functions which have their own subscription. The problem with this approach is, that i can’t

execute a query in a different thread. A workaround could be to open the db connection inside of the get_data_db function.

def open_db(status):
global conn # Verbindungsobjekt
global dbOpen
dbOpen = False
try:
print(“open_db: {}”.format(1))
conn = db.connect(’/home/robert/crossbar_setting/Autobahn_Test/node2/python/sqlite/berlin_building.sqlite’) # Oeffnen einer Datenbankverbindung
conn.enable_load_extension(True) # Das einladen von extension aktivieren
cur = conn.cursor() # Erzeugen eines cursors mit dem SQL-Statements ausgefuerht werdne koennen
cur.execute(“select load_extension(‘libspatialite.so’)”) # Einladen der libspatialite.so
conn.enable_load_extension(False) # Funktionalitaet zum Einladen von extensions wieder deaktivieren
conn.row_factory = dict_factory # Festlegen wie ein Ergebnis formatiert werden soll
cur.close() # Cursor wieder schliessen
dbOpen = True
except:
dbOpen = False
self.publish(u’de.db.isopen’, dbOpen)

sub = yield self.subscribe(open_db, ‘de.open.db’)
print(“subscribed to topic ‘open_db’: {}”.format(sub))

``

def close_db(status):
print(“close_db: {}”.format(1))
dbClosed = False
try:
conn.close()
dbClosed = True
dbOpen = False
except:
dbClosed = False
self.publish(u’de.db.isclosed’, dbClosed)

sub = yield self.subscribe(close_db, ‘de.close.db’)
print(“subscribed to topic ‘close_db’: {}”.format(sub))

``

def get_data_db(queryData):
print(“get_data_db: {}”.format(queryData))

“”"
Idee:
Eine Liste mit allen Informationen wird zusammengestellt.
die letztem drei Eintraege beinhaltet die tilenummer, vor der tilenummer kommt der tabellenname.
Alle weiteren Eintraege stellen Spaltennamen dar. Der erste Eintrag beinhaltet den Spaltennamen
id, wenn in der Tabelle vorhanden.
[“id”, “column2”, “column3”, “columnx”, “tabellenname”, “zoom”, “xtile”, “ytile”]
Aus diesen Informationen wird das Select-Statement konstruiert.
“”"

Den String splitten

splitList = queryData.split("/")

Anzahl an Elementen in der Liste ermitteln

listCount = len(splitList)

Tilenummer extrahieren und neu zusammensetzen

tileNumber = splitList[listCount-3] + “/” + splitList[listCount-2] + “/” + splitList[listCount-1]

print(tileNumber)

BBOX berechnen

maputil = MapUtil()
bbox = maputil.num2deg(str(tileNumber)) # Umwandeln der TileNummer zu einer BoundingBox

Tabellennamen aus der Liste extrahieren

tabellenname = splitList[listCount-4]

Alle Spalten aus der Liste extrahieren und in einer neuen Liste abspeichern

spalten = splitList[0:listCount-4]

Anzahl an Spalten ermitteln

listCount = len(spalten)

spaltenSQL = “”
forCounter = 0

Zusammensetzen eines String mit allen Spalten

for spalte in spalten:
spaltenSQL += spalte
if(forCounter < listCount-1):
spaltenSQL += “,”
forCounter += 1

Zusammensetzen aller Informationen zu einem SQL-Befehl

cmd = “”"
SELECT asgeojson(CastToPolygon(st_intersection(BuildMBR({2}, {3}, {4}, {5}), Geometry))) as geojson, {0}
FROM {1}
WHERE MBRIntersects(BuildMBR({2}, {3}, {4}, {5}), Geometry);""".format(spaltenSQL, tabellenname, bbox[0], bbox[1], bbox[2], bbox[3])

cur = conn.cursor()
cur.execute(cmd)

result = cur.fetchall()
cur.close()

featureCollection = list()

for row in result:
geom = geojson.loads(row[‘geojson’])
row.pop(‘geojson’)
#idBuilding = geojson.loads(row[‘id’])
#row.pop(‘id’)
feature = geojson.Feature(geometry = geom, properties = row)

featureCollection.append(feature)

finalFeatureCollection = geojson.FeatureCollection(featureCollection)
geoJsonDump = geojson.dumps(finalFeatureCollection, sort_keys=True)

print(geoJsonDump)

return geoJsonDump
#reactor.callLater(0, dbThread, geoJsonDump)

reg = yield self.register(get_data_db, ‘de.map.data.request’)
print(“Funktion registriert: get_data_db”)

``

···

Am Mittwoch, 26. August 2015 18:56:33 UTC+2 schrieb Tobias Oberstein:

We need to see what you do in the get_data_db procedure. You must not block the reactor … eg need to use appropriate db code as db drivers are blocking most often

Sent from Mobile (Google Nexus 5)

Am 26.08.2015 17:30 schrieb “Robert Gregat” rgr...@gmail.com:

Hello everyone,
I use Crossbar.io together with Autobahn|Python Twisted on my Raspberry Pi 2 (Ubuntu Mate).

For some testing i query a sqlite/spatialite-db from an angularJS-App. For the communication i use angular-wamp.

So I have a procedure in Python which do all the heavy work (prepare the query, execute it and produce a geojson-string from the result).

My idea behind this is to have something similar like a vector-tile query based on slippy-map Tiles (zoom/xtile/ytile).

Everything works fine, dispite of one thing. If i execute the call 2-3 or more times i get my response after all calls are finished.

On my Pi 2 i did some console.log() to see what happened inside of the procedure. The procedure works normal and execute one by one, but on the client side i get every result after every call is processed at once.

My question is, when my procedure execute one by one, why did i get the response only after no more execution are going on?

Stripped down python code:

class AppSessionSpatialite(ApplicationSession):

@inlineCallbacks

def onJoin(self, details):

def get_data_db(queryData):

return result

yield self.register(get_data_db, ‘de.map.data.request’)

You received this message because you are subscribed to the Google Groups “Crossbar” group.

To unsubscribe from this group and stop receiving emails from it, send an email to crossbario+...@googlegroups.com.

To post to this group, send email to cros...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/crossbario/b89d8d7c-ff1d-4612-a1ad-d07c4a794985%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

0 Likes