steve [ARCHIVE] on Nostr: 📅 Original date posted:2012-07-24 📝 Original message:Hi Michael, from what I ...
📅 Original date posted:2012-07-24
📝 Original message:Hi Michael,
from what I have noticed, bitcoin blockchain download/verfication all
happens in 1 thread. (so multicores doesnt really help)
That said, I have never tried on an ssd.
What I do have is 6 SATA 6gbs configed as RAID0 Drives.
32gb of ram. ubuntu 64 (yeah I know), this runs upto 16 VM's
(I have 4 of these)
However I have not tried to download the blockchain on the master os,
just in virtulisation. However, the dedicated machines that I have been
using for benchmarking the VM's against is a q6600 8gb ram sata2 hdd -
Win 7 (seems faster than slackware...) to me it has always felt like
network bandwidth was the issue. I might instrument the bitcoin-qt exe
to only pick low ping nodes (has someone already done this?)
I guess it is time to start some benchmarking (like the gpu comparison page)
hte verification for the 5 past 5 days was negliglable. I am off on a
flight to australia tomorrrow, so I will set some breakpoints and do
some timings in a debugger.
This will all happen on an e-450 (wonderful machine!)
Thanks very much for your response. it would seem that I am 'doing it
wrong' :/
cheers mate,
steve
(this message isnt signed because I have forgotten my password.)
On 24/07/2012 09:25, Michael Grønager wrote:
> Hi Steve,
>
> 45-90 minutes - note that its numbers from March/April, so a bit
> longer today, but far, far away from the 12 hours.
>
> I am using libcoin and the bitcoind build based on this. Libcoin is
> based on the Satoshi client, but refactured to use an async
> concurrency model. I also did a minor tweeks to the db parameters. It
> has earlier been tested up against Satoshi bitcoin where on some
> OS'es it performs similarly (at least on some linuxes) and on some
> faster (e.g. mac).
>
> What is your CPU load during a block download ? (both initially/up to
> the point where verification sets in and after). The initial download
> is typically disk I/O bound, the verification stage CPU bound, though
> I lean to believe that even there it is disk I/O bound (at least on
> my system ~50% CPU load). What should be better in libcoin is the
> concurrency model. The Satoshi client uses a pure reentrant mutexes
> model, that is not generally believed to motivate the best coding
> practice nor performance, you might end up without the concurrency
> you initially strived for *). As mentioned earlier libcoin uses a
> pure async concurrency model (and so does libbitcoin btw).
>
> I would like to stress again that these numbers will depend largely
> on the system running the test - I would call my laptop a bit over
> the average today (MB Pro, 2.66Ghz i7 dual core, 8GBRAM, 512GB SSD).
> But again 12 hours - I only reach such numbers on some of my VPS'es
> (linode 1024) that are known for notoriously slow disk I/O. (here I
> have a few % CPU load during the verification indicating indeed that
> the disk i/o is the culprit).
>
> Cheers,
>
> Michael
>
>
> *) I like this Dave Butenhof quote: "The biggest of all the big
> problems with recursive mutexes is that they encourage you to
> completely lose track of your locking scheme and scope. This is
> deadly. Evil. It's the "thread eater". You hold locks for the
> absolutely shortest possible time. Period. Always. If you're calling
> something with a lock held simply because you don't know it's held,
> or because you don't know whether the callee needs the mutex, then
> you're holding it too long. You're aiming a shotgun at your
> application and pulling the trigger. You presumably started using
> threads to get concurrency; but you've just PREVENTED concurrency."
>
>
>
>
> On 23/07/2012, at 17:54, steve wrote:
>
> Hi Michael,
>
> On 23/07/2012 10:00, Michael Grønager wrote:
>>>> I get a full blockchain from scratch in 45 minutes on my
>>>> laptop, /M
>>>>
> Hang on a sec, in 45 minutes you can download the entire chain from
> the genesis block?
>
> I have been doing extensive testing in this area and would love to
> know what is special about your setup (I have never had the entire
> chain in under 12 hours, infact it is normally closerto 24.) I have
> an extensive setup of test machines, everything from e4300 to
> phenom2x6 to i5's.
>
> as an example on an amd e-450 with 4gb ram, and approx 3gb/s
> internet connection it took 2 hours to sync the last 5 days.
>
> Maybe i am missing something important...
>
> Any additional information that you could provide to help me with
> testing would be really appreciated.
>
> cheers,
>
> steve
>
>>
>> ------------------------------------------------------------------------------
>>
>>
Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond.
>> Discussions will include endpoint security, mobile security and the
>> latest in malware threats.
>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________ Bitcoin-development
>> mailing list Bitcoin-development at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
>
>
📝 Original message:Hi Michael,
from what I have noticed, bitcoin blockchain download/verfication all
happens in 1 thread. (so multicores doesnt really help)
That said, I have never tried on an ssd.
What I do have is 6 SATA 6gbs configed as RAID0 Drives.
32gb of ram. ubuntu 64 (yeah I know), this runs upto 16 VM's
(I have 4 of these)
However I have not tried to download the blockchain on the master os,
just in virtulisation. However, the dedicated machines that I have been
using for benchmarking the VM's against is a q6600 8gb ram sata2 hdd -
Win 7 (seems faster than slackware...) to me it has always felt like
network bandwidth was the issue. I might instrument the bitcoin-qt exe
to only pick low ping nodes (has someone already done this?)
I guess it is time to start some benchmarking (like the gpu comparison page)
hte verification for the 5 past 5 days was negliglable. I am off on a
flight to australia tomorrrow, so I will set some breakpoints and do
some timings in a debugger.
This will all happen on an e-450 (wonderful machine!)
Thanks very much for your response. it would seem that I am 'doing it
wrong' :/
cheers mate,
steve
(this message isnt signed because I have forgotten my password.)
On 24/07/2012 09:25, Michael Grønager wrote:
> Hi Steve,
>
> 45-90 minutes - note that its numbers from March/April, so a bit
> longer today, but far, far away from the 12 hours.
>
> I am using libcoin and the bitcoind build based on this. Libcoin is
> based on the Satoshi client, but refactured to use an async
> concurrency model. I also did a minor tweeks to the db parameters. It
> has earlier been tested up against Satoshi bitcoin where on some
> OS'es it performs similarly (at least on some linuxes) and on some
> faster (e.g. mac).
>
> What is your CPU load during a block download ? (both initially/up to
> the point where verification sets in and after). The initial download
> is typically disk I/O bound, the verification stage CPU bound, though
> I lean to believe that even there it is disk I/O bound (at least on
> my system ~50% CPU load). What should be better in libcoin is the
> concurrency model. The Satoshi client uses a pure reentrant mutexes
> model, that is not generally believed to motivate the best coding
> practice nor performance, you might end up without the concurrency
> you initially strived for *). As mentioned earlier libcoin uses a
> pure async concurrency model (and so does libbitcoin btw).
>
> I would like to stress again that these numbers will depend largely
> on the system running the test - I would call my laptop a bit over
> the average today (MB Pro, 2.66Ghz i7 dual core, 8GBRAM, 512GB SSD).
> But again 12 hours - I only reach such numbers on some of my VPS'es
> (linode 1024) that are known for notoriously slow disk I/O. (here I
> have a few % CPU load during the verification indicating indeed that
> the disk i/o is the culprit).
>
> Cheers,
>
> Michael
>
>
> *) I like this Dave Butenhof quote: "The biggest of all the big
> problems with recursive mutexes is that they encourage you to
> completely lose track of your locking scheme and scope. This is
> deadly. Evil. It's the "thread eater". You hold locks for the
> absolutely shortest possible time. Period. Always. If you're calling
> something with a lock held simply because you don't know it's held,
> or because you don't know whether the callee needs the mutex, then
> you're holding it too long. You're aiming a shotgun at your
> application and pulling the trigger. You presumably started using
> threads to get concurrency; but you've just PREVENTED concurrency."
>
>
>
>
> On 23/07/2012, at 17:54, steve wrote:
>
> Hi Michael,
>
> On 23/07/2012 10:00, Michael Grønager wrote:
>>>> I get a full blockchain from scratch in 45 minutes on my
>>>> laptop, /M
>>>>
> Hang on a sec, in 45 minutes you can download the entire chain from
> the genesis block?
>
> I have been doing extensive testing in this area and would love to
> know what is special about your setup (I have never had the entire
> chain in under 12 hours, infact it is normally closerto 24.) I have
> an extensive setup of test machines, everything from e4300 to
> phenom2x6 to i5's.
>
> as an example on an amd e-450 with 4gb ram, and approx 3gb/s
> internet connection it took 2 hours to sync the last 5 days.
>
> Maybe i am missing something important...
>
> Any additional information that you could provide to help me with
> testing would be really appreciated.
>
> cheers,
>
> steve
>
>>
>> ------------------------------------------------------------------------------
>>
>>
Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond.
>> Discussions will include endpoint security, mobile security and the
>> latest in malware threats.
>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________ Bitcoin-development
>> mailing list Bitcoin-development at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
>
>