How long are the Work Units going to be?
Message boards : Number crunching : How long are the Work Units going to be?
| Author | Message | |
|---|---|---|
|
My first batch of 100 WU's have all been given a Boinc time of 8.23 minutes and the 17 I have done all took about this time (Successful but with an error).
|
||
ID:
194 | Rating: 0
| rate:
/
|
||
Andre Kerstens
Forum moderator Project tester Volunteer tester 
Joined: Sep 11 06 Posts: 749 ID: 1 Credit: 15,199 RAC: 0 |
||
|
I've tested the new WUs this afternoon and they estimate and run as 1.5 hours on a P4 3.2 GHz. We will deploy this evening.
My first batch of 100 WU's have all been given a Boinc time of 8.23 minutes and the 17 I have done all took about this time (Successful but with an error). |
||
|
ID:
195 | Rating: 0
| rate:
/
|
||
|
I've tested the new WUs this afternoon and they estimate and run as 1.5 hours on a P4 3.2 GHz. We will deploy this evening. I think your estimates are way too short. I just completed 2 of the new 5.02 WUs and they both ran over 4 hours on an XP2600. |
||
|
ID:
226 | Rating: 0
| rate:
/
|
||
|
I got about 8 hours on a P4/1.6GHz, which should have gotten around 3 hours, eh?
|
||
|
ID:
237 | Rating: 0
| rate:
/
|
||
|
AK wrote:
I've tested the new WUs this afternoon and they estimate and run as 1.5 hours on a P4 3.2 GHz. We will deploy this evening. Is yours a single CPU ? I started on 5.02 earlier today No downlaod problems. On Windoze at 3.06GHz HT running 1 CPU charmm 5.02 took 4 hrs 9 mins. That was down from 5 hrs 30 mins for charmm 5.01 on this host. It just occured to me, that actually that is only using 50% of my CPU. So if I run both CPU's I can crank out 2 in about the same time, so effective wu time is about 2 hours. Higher but not as far off your estimate as being more than double. I just cranked my 3.06GHz HT and 3.8GHz HT up to 2 CPUs to see the results. I had been not running boinc on them up until Sept 1st, put out too much heat into my room which requires extra A/C for me to be comfortable, they can take it but I can't. I was not going to run both CPUs until Oct 1st, when it cools off more, but I'll test it for the week-end. 4 wu download now, no problems. |
||
|
ID:
239 | Rating: 0
| rate:
/
|
||
|
Andre Kerstens
Forum moderator Project tester Volunteer tester
Joined: Sep 11 06 Posts: 749 ID: 1 Credit: 15,199 RAC: 0 |
||
|
Hi Keith,
|
||
|
ID:
240 | Rating: 0
| rate:
/
|
||
|
Hi Keith, My 3 results: http://docking.utep.edu/result.php?resultid=8591 http://docking.utep.edu/result.php?resultid=8594 http://docking.utep.edu/result.php?resultid=8910 For my 3.0GHz P4, looks like it averages between 17,000 - 18,000 seconds. |
||
|
ID:
242 | Rating: 0
| rate:
/
|
||
|
For dual-cored machine when running both cores, we can usually see 85-95% of speed on each. There is no need (nor possibility) of changing computation estimation on dual/multi cores or CPUs system.
|
||
|
ID:
251 | Rating: 0
| rate:
/
|
||
|
Andre wrote:
Yes, both of the machine I got the estimate from are single cpu. And even if they were dual cpus, boinc would schedule only job on one cpu (2 running at a time). Yes, that is true But what I meant is I can run 2 results at the same time, both completing in the same 4.5 hours approx. I turn out 2 in that time. while a single CPU turns out 1 at a time. If a single CPU took 2.25 hours each it too would turn out 2 in the same time, just they were processed 1 after the other and not simultaneously. So effectively a HT CPU taking 4.5 hours for two results has an equivilemnt of 2.25 single CPU hours per result. If only running 1 thread then a HT CPU is really only running at 50% of its capacity, not 50% speed. Anyway last nights sucessful results for charmm 5.02. Windoze on 3.8GHz HT completed 4 at average 4hr 26mins each (4:25 to 4:29) fairly even Windoze on 3.06GHz HT completed 4 at average 4hr 37min each (4:09 to 5:06) |
||
|
ID:
264 | Rating: 0
| rate:
/
|
||
|
If only running 1 thread then a HT CPU is really only running at 50% of its capacity, not 50% speed.I can't agree there. HT is not a dual-core, nor dual-CPU...actually far from it. If you are running without HT (or running only single thread/application), you are at worst loosing 25% of total performance. Older Xeon are known to do poor on HT and there are cases when running both threads on HT actually makes whole system running slower and overall performance goes down (due to high latencies to CPU cache and memory, all running via FSB etc.); for example on highly optimalized code like Einstein akosf's optimalization was. You may find some interesting vaules in my BOINC table . (I have been running Intel HT machine but abandomed them 1+ years ago when I went dual-cores and again upgraed recently to Conroes). |
||
|
ID:
275 | Rating: 0
| rate:
/
|
||
|
If only running 1 thread then a HT CPU is really only running at 50% of its capacity, not 50% speed.I can't agree there. Haha Honza, I also have a table running ;) Can you send me the conroe at home :p Next time I will have one. PS: Good table so far, better overview than mine :) |
||
|
ID:
277 | Rating: 0
| rate:
/
|
||
|
I have dopenloaded 116 WUs on my Linux machine (AMD 3000+). The cruniching time is about 4 minutes. All WUs stay on pending. |
||
|
ID:
295 | Rating: 0
| rate:
/
|
||
|
I have dopenloaded 116 WUs on my Linux machine (AMD 3000+). The cruniching time is about 4 minutes. All WUs stay on pending. Your results are not "finishing" - they are failing. <core_client_version>5.5.0</core_client_version> <stderr_txt> Calling BOINC init. Starting charmm run... ERROR - Charmm exited with code 1. Calling BOINC finish. </stderr_txt> |
||
|
ID:
298 | Rating: 0
| rate:
/
|
||
|
I have dopenloaded 116 WUs on my Linux machine (AMD 3000+). The cruniching time is about 4 minutes. All WUs stay on pending. I am also getting the same error after doing 50 odd WU's, all done in less than 300 seconds. I believe I may still doing the 5.01 WU's. I dumped over 100 of these when Andre asked us to but I then downloaded 5.01 WU's again. At this rate I will get rid of then fairly quickly and can start on the 5.02 WU's. I am running Linux on a AMD Opteron with dual 848 cpus. |
||
|
ID:
311 | Rating: 0
| rate:
/
|
||
|
Andre Kerstens
Forum moderator Project tester Volunteer tester
Joined: Sep 11 06 Posts: 749 ID: 1 Credit: 15,199 RAC: 0 |
||
|
OK;maybe this is not completely clear yet. The windows fix was an app fix and therefore the windows app version went to 5.2. The linux and mac version of the app are still at 5.01. because the error -131 fix was only a fix to the input file of the app (no version change). We are currently working on the 'charmm exited with error 1' problem, which turns out to be much harder, because not all linux machines have this problem, only some.
I have dopenloaded 116 WUs on my Linux machine (AMD 3000+). The cruniching time is about 4 minutes. All WUs stay on pending. |
||
|
ID:
314 | Rating: 0
| rate:
/
|
||
|
Just out of interest on what i should attach to this project.
|
||
|
ID:
1396 | Rating: 0
| rate:
/
|
||
|
> @ clownius,
|
||
|
ID:
1399 | Rating: 0
| rate:
/
|
||
|
> @ clownius, I will add Barton 3200+ (2.2 Ghz) - 6h28min (Win2k) |
||
|
ID:
1408 | Rating: 0
| rate:
/
|
||
|
suguruhirahara
Forum moderator Volunteer tester 
Joined: Sep 13 06 Posts: 282 ID: 15 Credit: 56,614 RAC: 0 |
||
|
My linux machine completed 15% or so, though 7 hours are spent... It seems too slow. What's wrong?:(
|
||
|
ID:
1411 | Rating: 0
| rate:
/
|
||
|
|
||
|
ID:
1418 | Rating: 0
| rate:
/
|
||
|
David Ball
Forum moderator Volunteer tester 
Joined: Sep 18 06 Posts: 274 ID: 115 Credit: 1,634,401 RAC: 0 |
||
|
My linux machine completed 15% or so, though 7 hours are spent... It seems too slow. What's wrong?:( I looked at your Linux machine and noticed a couple of things. 1. It only has 512 MB of ram and it could be that something else running on the machine has caused it to reach the point where it's doing a lot of swapping to disk. From the Linux command line, what do you get if you type the command "free" ? Also, on many machines with built in graphics and even some with graphics cards, some of the ram memory is used by the display. Have you recently gone to a larger screen resolution, which would take more memory? Bringing the machine up to 1 GB of ram would probably do wonders. 2. It's a Pentium D, which is dual core, but it says it has 1 CPU. Linux should report dual core as 2 CPU's. The introduction of dual core chips has really confused the terminology. They're really 2 CPU's on one chip, with a little shared circuitry. Most software (like Microsoft windows) licenses software by how many physical chips are plugged into CPU sockets, therefore it's called 1 CPU but 2 cores. Even though Windows says it's 1 CPU for licensing purposes, the operating system has to treat it as 2 CPU's. You'll even see separate graphs for each CPU in the Windows performance monitor. Linux also has to treat it as 2 CPU's and that means that you need the multiple CPU (or multiple core) Linux kernel, which has extra code in it that is not included in the single CPU (Single core) Linux Kernel. Both the single CPU and multiple CPU Linux kernels should have come with your distribution, including some variations on both, such as the "hugemem" kernels, if you have a LOT of memory. You don't need the "hugemem" variation, just the "smp" variation. How many CPU's does the command line program "top" say your system has? Maybe it's just BOINC that is reporting your system as single CPU. I tried to find info on your distribution, but I can't speak or read Japanese and most of the Vine Linux site was only in Japanese. Hope this helps, -- David |
||
|
ID:
1419 | Rating: 0
| rate:
/
|
||
|
suguruhirahara
Forum moderator Volunteer tester
Joined: Sep 13 06 Posts: 282 ID: 15 Credit: 56,614 RAC: 0 |
||
|
Thanks david:)
1. It only has 512 MB of ram and it could be that something else running on the machine has caused it to reach the point where it's doing a lot of swapping to disk. From the Linux command line, what do you get if you type the command "free" ? Also, on many machines with built in graphics and even some with graphics cards, some of the ram memory is used by the display. Have you recently gone to a larger screen resolution, which would take more memory? Bringing the machine up to 1 GB of ram would probably do wonders. It says that 220/512MB is used, and no swap is used. So this issue isn't due to lack of RAM. 2. It's a Pentium D, which is dual core, but it says it has 1 CPU. Linux should report dual core as 2 CPU's. The introduction of dual core chips has really confused the terminology. They're really 2 CPU's on one chip, with a little shared circuitry. Most software (like Microsoft windows) licenses software by how many physical chips are plugged into CPU sockets, therefore it's called 1 CPU but 2 cores. Even though Windows says it's 1 CPU for licensing purposes, the operating system has to treat it as 2 CPU's. You'll even see separate graphs for each CPU in the Windows performance monitor. Linux also has to treat it as 2 CPU's and that means that you need the multiple CPU (or multiple core) Linux kernel, which has extra code in it that is not included in the single CPU (Single core) Linux Kernel. Both the single CPU and multiple CPU Linux kernels should have come with your distribution, including some variations on both, such as the "hugemem" kernels, if you have a LOT of memory. You don't need the "hugemem" variation, just the "smp" variation. How many CPU's does the command line program "top" say your system has? Maybe it's just BOINC that is reporting your system as single CPU. I tried to find info on your distribution, but I can't speak or read Japanese and most of the Vine Linux site was only in Japanese. Thanks for explanation in detail:) The linux in fact runs on virtual machine VMWare and crunches a workunit, and also windows, its host OS, crunches another workunit at the same time. Hence two cores on 1 CPU is actively used. Also I'd love to test Windows besides Linux:) As the Japanese linux distro is improved with a view of stability, the upgrade option is relatively stricted and conservative. There are yet unavailable to get smp kernel 2.6 for this version of vine linux 4.0. If Kentsfield weren't so expensive, I would buy one and seek for other distro and install kernel-smp! YET workunits on the linux are being computed and progressed well now, though I don't know why. Hmm... it seems whether workunits are crunched well or not depends on the virtual machine's condition:( I let it crunch workunits as many as possible until it will go bad/mad;-) ____________ 
I'm a volunteer participant; my views are not necessarily those of Docking@Home or its participating institutions. |
||
|
ID:
1421 | Rating: 0
| rate:
/
|
||
|
At least, i'm not the only one now with the 'progress bar' =0 and the 'time to completion' static. |
||
|
ID:
1424 | Rating: 0
| rate:
/
|
||
|
David Ball
Forum moderator Volunteer tester
Joined: Sep 18 06 Posts: 274 ID: 115 Credit: 1,634,401 RAC: 0 |
||
|
Oh, that makes a lot of difference. I've never run VMWare, but I think most virtual environments are limited to 1 CPU for the client OS, no matter how many the host computer has. Keeping in mind that I've never run VMWare, it might have been swapping. If the host OS (Windows) decided to swap some of the memory allocated to the VMWare virtual machine, the client OS (Linux) might not know about it. As far as I know, the Client OS thinks it has 512 MB of ram and some swap space on disk. I don't think the client OS would know if part of it's ram was really on disk in the host OS swap space. On some of the virtual machine managers that run on Linux and only host Linux virtual machines, they have both a guaranteed and a burstable memory limit on the client Linux VPS (Virtual Private Servers running a Linux client OS). From what I can tell, the virtual machine manager allows the sum of the "Guaranteed" ram for the clients to equal the host Linux machines ram + swap space. In other words, a host machine with 2 GB of ram and 4 GB of swap space can host several VPS with a total for all the virtual Linux machines "Guaranteed" ram being about 6 GB. Things get really confusing when Virtual Machines are involved. I have run a VPS, but it's been months since I had access to it so I've forgotten exactly what top and free reported on the client OS. Since it was Linux hosted on Linux (that's all the virtual environment supported), it was probably different from Linux hosted on Windows. You might have been able to choose to run a different client version of Linux than the host Linux machine but the VPS I had was running the same for both host and client. It was more of a partitioned environment rather than a true virtual machine, but you could reboot a VPS without having any effect on the other VPSs running on the same host. I don't know how it changed if the client Linux was a different version but I think that was possible (not sure on this point). Virtual machines do some weird stuff!!! Also, Windows has a LOT of OS processes running in the background, which results in a lot of context switches that flush a lot of the CPU cache memory. Is there any chance that the host machine was running a virus scan or something like that when the work unit was running so slow. One of these days, I hope to get around to trying VMWare. It sounds like you've done a good job of getting it to work. How hard was it to install VMWare and setup Linux as a client OS? BTW, I have 2 machines that should be roughly equivalent running Docking @ Home. The Windows XP machine ( Socket 478 Celeron 2.3 GHz with 128 KB l2 cache and 1.5 GB DDR Ram ) takes about 10 hours and 20 minutes of CPU time to run a Docking work unit. The Linux machine ( Socket A Sempron 2500+ with 256KB cache and 1 GB DDR ram on Fedora core 3 with a 2.6 kernel ) takes about 4 hours and 8 minutes of CPU time to run a Docking work unit. That's a Socket A Sempron so it uses a FSB (Front Side Buss) like the old Duron and Athlon chips instead of having a memory controller like the modern Semprons. For some reason, Climate Prediction takes almost exactly the same time on both machines (3.9 seconds per timestep). I guess it depends on what kind of code you're running. I'm almost certain that Linux does lazy writes to disk (puts it in a buffer and flushes the buffer about every 5 seconds or earlier if it needs the memory). I think Windows defaults to doing the disk write immediately and then returning to the program doing the write. With all the debug info being written to disk in the current Docking application, I suspect that is why it takes so much longer on windows. The work units use cumulative CPU time though, not cumulative real time so the extra CPU time for the docking work units on windows would probably be caused by: 1. Windows counts some or all of the time used in the disk write as CPU time used? 2. By not doing a Lazy write, windows does a context switch for each disk write and that (plus running the actual disk write code) causes the cache to get flushed. 3. Both of the above. Note that Climate Prediction, which takes about the same time on both machines, only does a disk write every few minutes. I hope this makes sense. These are confusing subjects, even for an old x86 assembler language programmer like myself. I try to keep up with the various cpu architectures when I have the time and I read a lot of the linux kernel mailing lists, but these days I don't have as much time for programming and most of what I do program is in c/c++. I still have the compiler produce an assembly language listing for me when I'm trying to optimize something though. It's amazing how a subtle difference can speed up code, especially with out-of-order execution and cache line alignment, etc. The architecture of modern CPUs is amazing!!! Regards, -- David |
||
|
ID:
1427 | Rating: 0
| rate:
/
|
||
|
David, I appreciate for your detailed explanations of things. You have a natural knack for being able to break complicated subjects down into simpler terms that even I can understand. There are a few other folks around here that are also good at troubleshooting/explaining technical issues and happy to help out fellow crunchers; and to all of you, I'd like to say, "Thanks for taking the time."
|
||
|
ID:
1429 | Rating: 0
| rate:
/
|
||
|
suguruhirahara
Forum moderator Volunteer tester
Joined: Sep 13 06 Posts: 282 ID: 15 Credit: 56,614 RAC: 0 |
||
|
Agree with miss booty:) Thanks for tough work!
Keeping in mind that I've never run VMWare, it might have been swapping. If the host OS (Windows) decided to swap some of the memory allocated to the VMWare virtual machine, the client OS (Linux) might not know about it. As far as I know, the Client OS thinks it has 512 MB of ram and some swap space on disk. I don't think the client OS would know if part of it's ram was really on disk in the host OS swap space. I'm confusing on this point when I see Windows task manager. It says that vmware-vmx is just using 3.4MB, while charmm on windows is using 6.5MB. Yet the manager says that physical memory is used 80%. In addition, System monitor showed that 0 bytes is used, whereas 138/512MB RAM is used. Also my host OS windows is running without stress, so I think swap isn't used so heavily that charmm is run on swap. but it exists lag between the time spent and that really used to crunch. I'm puzzled in many pointa. One of these days, I hope to get around to trying VMWare. It sounds like you've done a good job of getting it to work. How hard was it to install Vmware and setup Linux as a client OS? It's easy for you, and you can get install any distro you want, with some config files and ISO image of the distribution such as fedora, debian. But you'll need many RAM if you'd like to install SUSE, which I tried to install but the host OS has freezed. There are plenty of install guide on the net, so it's not hard for you to get manuals of "vmware player" I guess it depends on what kind of code you're running. I'm almost certain that Linux does lazy writes to disk (puts it in a buffer and flushes the buffer about every 5 seconds or earlier if it needs the memory). I think Windows defaults to doing the disk write immediately and then returning to the program doing the write. With all the debug info being written to disk in the current Docking application, I suspect that is why it takes so much longer on windows. I'm not sure on this point, but charmm for linux is ver 5.2 and disk writing process is probably different from each charmm. I hope this makes sense. These are confusing subjects, even for an old x86 assembler language programmer like myself. I try to keep up with the various cpu architectures when I have the time and I read a lot of the linux kernel mailing lists, but these days I don't have as much time for programming and most of what I do program is in c/c++. I still have the compiler produce an assembly language listing for me when I'm trying to optimize something though. It's amazing how a subtle difference can speed up code, especially with out-of-order execution and cache line alignment, etc. The architecture of modern CPUs is amazing!!! As you may know by readin a thread on docking@home science, the science app named charmm is written in FORTRAN, not c/c++, and according to Dr Andre, the system has been improved for many years around 20 years. I'm not quite sure on this point, but I feel there isn't large room for them to make a quite radical change on the application. ____________
I'm a volunteer participant; my views are not necessarily those of Docking@Home or its participating institutions. |
||
|
ID:
1439 | Rating: 0
| rate:
/
|
||
|
Just out of interest on what i should attach to this project. Might be a little late for this discussion but a 700mhz system is a PIII class? If so, there have been some issues with validating results. The CPU will handle the work just fine but you may get no credit for it. Last I heard it's still being worked on. The project staff could give you more detailed information. ____________ KWSN - A Shrubbery http://www.kwsnforum.com |
||
|
ID:
1452 | Rating: 0
| rate:
/
|
||
|
VMware can work with up to two CPUs, provided that the physical hardware has two or more CPUs (dual core, two CPUs, or even single P4 with Hyperthreading). |
||
|
ID:
1454 | Rating: 0
| rate:
/
|
||
|
suguruhirahara
Forum moderator Volunteer tester
Joined: Sep 13 06 Posts: 282 ID: 15 Credit: 56,614 RAC: 0 |
||
|
Yes, but it needs to purchase VMWare ESX Server to run smp kernel. If one use VMWare player, the number of available CPU is limited to 1. ____________
I'm a volunteer participant; my views are not necessarily those of Docking@Home or its participating institutions. |
||
|
ID:
1460 | Rating: 0
| rate:
/
|
||
|
Oh well... I use a not-exactly-purchased Vmware Workstation.. shh! >:) |
||
|
ID:
1482 | Rating: 0
| rate:
/
|
||
Message boards : Number crunching : How long are the Work Units going to be?
Database Error: The MySQL server is running with the --read-only option so it cannot execute this statement
array(3) {
[0]=>
array(7) {
["file"]=>
string(47) "/boinc/projects/docking/html_v2/inc/db_conn.inc"
["line"]=>
int(97)
["function"]=>
string(8) "do_query"
["class"]=>
string(6) "DbConn"
["object"]=>
object(DbConn)#35 (2) {
["db_conn"]=>
resource(150) of type (mysql link persistent)
["db_name"]=>
string(7) "docking"
}
["type"]=>
string(2) "->"
["args"]=>
array(1) {
[0]=>
&string(50) "update DBNAME.thread set views=views+1 where id=31"
}
}
[1]=>
array(7) {
["file"]=>
string(48) "/boinc/projects/docking/html_v2/inc/forum_db.inc"
["line"]=>
int(60)
["function"]=>
string(6) "update"
["class"]=>
string(6) "DbConn"
["object"]=>
object(DbConn)#35 (2) {
["db_conn"]=>
resource(150) of type (mysql link persistent)
["db_name"]=>
string(7) "docking"
}
["type"]=>
string(2) "->"
["args"]=>
array(3) {
[0]=>
object(BoincThread)#3 (16) {
["id"]=>
string(2) "31"
["forum"]=>
string(1) "2"
["owner"]=>
string(3) "100"
["status"]=>
string(1) "0"
["title"]=>
string(40) "How long are the Work Units going to be?"
["timestamp"]=>
string(10) "1163894061"
["views"]=>
string(4) "1889"
["replies"]=>
string(2) "29"
["activity"]=>
string(20) "9.8572744439152e-128"
["sufferers"]=>
string(1) "0"
["score"]=>
string(1) "0"
["votes"]=>
string(1) "0"
["create_time"]=>
string(10) "1158276489"
["hidden"]=>
string(1) "0"
["sticky"]=>
string(1) "0"
["locked"]=>
string(1) "0"
}
[1]=>
&string(6) "thread"
[2]=>
&string(13) "views=views+1"
}
}
[2]=>
array(7) {
["file"]=>
string(63) "/boinc/projects/docking/html_v2/user/community/forum/thread.php"
["line"]=>
int(184)
["function"]=>
string(6) "update"
["class"]=>
string(11) "BoincThread"
["object"]=>
object(BoincThread)#3 (16) {
["id"]=>
string(2) "31"
["forum"]=>
string(1) "2"
["owner"]=>
string(3) "100"
["status"]=>
string(1) "0"
["title"]=>
string(40) "How long are the Work Units going to be?"
["timestamp"]=>
string(10) "1163894061"
["views"]=>
string(4) "1889"
["replies"]=>
string(2) "29"
["activity"]=>
string(20) "9.8572744439152e-128"
["sufferers"]=>
string(1) "0"
["score"]=>
string(1) "0"
["votes"]=>
string(1) "0"
["create_time"]=>
string(10) "1158276489"
["hidden"]=>
string(1) "0"
["sticky"]=>
string(1) "0"
["locked"]=>
string(1) "0"
}
["type"]=>
string(2) "->"
["args"]=>
array(1) {
[0]=>
&string(13) "views=views+1"
}
}
}
query: update docking.thread set views=views+1 where id=31