Support Board
Date/Time: Sun, 24 Nov 2024 16:55:08 +0000
Post From: Allow Disabling of Compression or Do It in Another Thread
| [2013-11-18 06:58:12] |
| bfalk - Posts: 33 |
|
I really appreciate the positive response. I'm quite a nut about optimization and performance (even sometimes when unnecessary) which is one of the reasons I use SC and not some other bloated .NET software. I will try out the prerelease tomorrow, as I'm heading off to sleep. The previous benchmark I did was all done on a ramdisk, so disk I/O was not an issue there and I'm certainly not memory or CPU constrained. I'm also well aware of everything in my kernel FS filter stack, and there is nothing stalling my I/O in there (no AV, no windows defender, etc). Hopefully the prerelease gives better results! For comparison, I have a small piece of C that I wrote up a while back that downloads and writes all of @CD# (tick level, 3,782,500 points) in 16.984 seconds. I can pull with multiple threads increasing performance until I finally saturate my internet bandwidth. I can send you the source if you're interested. One thing that I would like to see in the DTC Bridge version of IQFeed would be parallel pulling of multiple symbols. One issue with IQFeed (I've brought it up with them previously) is that they fetch on a single thread, meaning iqconnect bottlenecks on decompression on the CPU and not download speed (3.2GHz single core gives about 10mbits/s of throughput). However, IQFeed puts each unique socket connection on it's own thread, meaning if you open one session for each symbol you can saturate bandwidth. Once again, I greatly appreciate the response. Thanks, and have a great day! |