Increasing publishing speeds

#1

Hi all,

I’m publishing large packets of data my publisher to the crossbar router (in Python, a large matrix the size of, say, 100x100x100). The transmission speeds are rather slow (~10-15 seconds), only to the central router at that. Currently, I’m creating a simple list of the flattened array to be passed into the ‘publication’. I’m new to the networking/async arena, so any help will be appreciated!

Thanks

0 Likes

#2

Hi!

If I understand correctly, then the matrix has 1 million elements. Even at 1 byte per element, that’s 1 MB, but since I assume you have data in there, it’s probably more.

What kind of network do you use for transmitting this?

With e.g. a slow consumer DSL connection @1 Mbit/s upload speed, even for 1 MB this could be in the range you describe.

Regards,

Alex

···

Am Dienstag, 13. Dezember 2016 19:37:10 UTC+1 schrieb Sulaiman Somani:

Hi all,

I’m publishing large packets of data my publisher to the crossbar router (in Python, a large matrix the size of, say, 100x100x100). The transmission speeds are rather slow (~10-15 seconds), only to the central router at that. Currently, I’m creating a simple list of the flattened array to be passed into the ‘publication’. I’m new to the networking/async arena, so any help will be appreciated!

Thanks

0 Likes

#3

So, I’m not sure the best way to answer ‘network’, but I can tell you all about it. I’m running a WS server using Crossbar that’s connecting all the subscribers (which is really just a browser in Javascript, connecting to a localhost TCP server) to the publisher (which is my Python script). Does that help? Thanks so much!

···

On Wednesday, December 14, 2016 at 1:06:44 PM UTC-5, Alexander Gödde wrote:

Hi!

If I understand correctly, then the matrix has 1 million elements. Even at 1 byte per element, that’s 1 MB, but since I assume you have data in there, it’s probably more.

What kind of network do you use for transmitting this?

With e.g. a slow consumer DSL connection @1 Mbit/s upload speed, even for 1 MB this could be in the range you describe.

Regards,

Alex

Am Dienstag, 13. Dezember 2016 19:37:10 UTC+1 schrieb Sulaiman Somani:

Hi all,

I’m publishing large packets of data my publisher to the crossbar router (in Python, a large matrix the size of, say, 100x100x100). The transmission speeds are rather slow (~10-15 seconds), only to the central router at that. Currently, I’m creating a simple list of the flattened array to be passed into the ‘publication’. I’m new to the networking/async arena, so any help will be appreciated!

Thanks

0 Likes