Shorter work units or BOINC gone silly?

My 2.4GHz Pentium 4 has suddenly decided to download over 30 work units. This is odd because:

1. My "connect to network every" is set to 0.1 days
2. I am using an unmodified, standard BOINC client (5.4.11) - no inflated benchmarks (if anything, they're very low for this PC)
3. The average turnaround on this PC is 2.27 days
4. Deadline for these WU is 26 November..

It seems to have happened right after BOINC ran its benchmarks, so perhaps it's a problem with BOINC. I've never seen it go this silly though:

Yes, the BOINC Client can do weird things @ times ... IMO

Heck, I've had Projects set to 0.0 Resource Share, No new work selected in the Client for that Project & still have had the project send me work ... 0_o

Yes, the BOINC Client can do weird things @ times ... IMO

Heck, I've had Projects set to 0.0 Resource Share, No new work selected in the Client for that Project & still have had the project send me work ... 0_o

It seems that many hosts have downloaded results more than needed / computed in time. So did my hosts. It's supposed that deadline is too short. On another thread named "No windows work?" I asked Andre to work on this issue...

thanks for reading,
suguruhirahara
____________

I'm a volunteer participant; my views are not necessarily those of Docking@Home or its participating institutions.

Your supercharged X6800 might have some hope to complete 33 WU in 5 days, but my poor old P4 hasn't got a hope ;D

Your supercharged X6800 might have some hope to complete 33 WU in 5 days, but my poor old P4 hasn't got a hope ;D

By my calculations for my X6800EE it should be able to do 75 in a 5 day time span, the E6600's I have attached aren't to far behind the X6800 though, they should be able to get over 65 done. They all downloaded about 50 WU's yesterday so I should be able to get them done with room to spare ...

I'll probably throw 2 or 3 more E6600's at the Project in the next week or so after I see how the WU's I have already work out ... :)

I'm bit afraid that these super good machines would get all workunits and prevent PIII machines from reproducing errors in order to find out a solution of the error...lol

Maybe this has to do with the fact that for some time you all got the 'commited to other platforms' message and the boinc client is trying to catch up whenever it gets a chance. It still doesn't make sense to download more work than ever can be completed before the deadline though... Can an experienced boincer explain a bit more about the work fetch policy maybe?

Andre

PS Deadline is 5 days currently.

It seems that many hosts have downloaded results more than needed / computed in time. So did my hosts. It's supposed that deadline is too short. On another thread named "No windows work?" I asked Andre to work on this issue...

thanks for reading,
suguruhirahara

Can an experienced boincer explain a bit more about the work fetch policy maybe?

Andre

PS Deadline is 5 days currently.

There is a bug in 5.4.11 and earlier where a task that is downloading does not count as present on the system, and therefore more work will be requested until some actually completes download.

This is fixed in the Alpha builds. When a task is comitted to the system, it is counted towards the work on the system, blocking downloads to the estimate of its processing time.

Sorry about the name butcher there John, at least I got the First name right anyway ... ;)

hundreds of wu ? been there done that (ufluids and sztaki not here yet)

called a 'work bomb' usualy happens because either

* the 5 or more last wu took under a minute
* the only wu's were short

nah JM VII's is probibly right

____________

This was a problem for many of us when Docking first opened alpha stage. My computer downloaded 300 WU's in the first few minutes after attatching to the project. BOINC (or my system, or some code in the app, whatever is responsible for making this judgement) initially interpreted the WUs to be 5 minutes long to crunch, as opposed to their more accurate time of 3 hours. Even though my "connect every _" was set to 0.1 days, the WUs just poured in to the point that my system was lagging (which is when I noticed it and shut it down). Most of us had no choice but to abort a lot of work.

The quota was set to 50/day after that, to prevent such a massive scale of downloads, but I am unaware of any other steps that devs may have taken to address this problem. Again, I'm not altogether sure where it stems from (server vs. client side, BOINC app, project app, etc.). Since this has only been an issue for first time users (after crunching a WU, the DCF normalizes and makes a more reliable estimation of the time it will take to complete the next one), and this project has been more or less closed to new users since the beginning, this problem has not crept up in awhile.

You can read more about it in the Too Many WUs! thread.

Atomic
____________
KWSN - Asylum for the Cynically Insane

This was a problem for many of us when Docking first opened alpha stage. My computer downloaded 300 WU's in the first few minutes after attatching to the project. BOINC (or my system, or some code in the app, whatever is responsible for making this judgement) initially interpreted the WUs to be 5 minutes long to crunch, as opposed to their more accurate time of 3 hours. Even though my "connect every _" was set to 0.1 days, the WUs just poured in to the point that my system was lagging (which is when I noticed it and shut it down). Most of us had no choice but to abort a lot of work.

The quota was set to 50/day after that, to prevent such a massive scale of downloads, but I am unaware of any other steps that devs may have taken to address this problem. Again, I'm not altogether sure where it stems from (server vs. client side, BOINC app, project app, etc.). Since this has only been an issue for first time users (after crunching a WU, the DCF normalizes and makes a more reliable estimation of the time it will take to complete the next one), and this project has been more or less closed to new users since the beginning, this problem has not crept up in awhile.

You can read more about it in the Too Many WUs! thread.

Atomic

We calculate the estimate based on the runtime of a result on a 3 GHz linux box. I assume that when the estimated runtime on my boinc manager is equal to the actual runtime, the FP estimate must be correct and will correctly scale to all other platforms. If that is not the case, could there be a problem in boinc? If this is not the correct way of getting a good FP estimate, please somebody step forward and tell us how to do this :-)

Thanks,
Andre

The developers set an estimate of the floating point operations required to complete a typical result. Therefore it is the server side where the cause of the problem lies.

We calculate the estimate based on the runtime of a result on a 3 GHz linux box. I assume that when the estimated runtime on my boinc manager is equal to the actual runtime, the FP estimate must be correct and will correctly scale to all other platforms. If that is not the case, could there be a problem in boinc? If this is not the correct way of getting a good FP estimate, please somebody step forward and tell us how to do this :-)

Thanks,
Andre

The developers set an estimate of the floating point operations required to complete a typical result. Therefore it is the server side where the cause of the problem lies.

Hi John,

Do you mean that if we would increase the FP estimate for our workunit, the estimate made by the boinc client would be better? (that would be easy to do)

Thanks

Andre

It seems that you picked a system that is a bit more efficient than average at these computations. Of the machines that I have attached, most are in the range of 5 to 6 which means that the calculations are taking 5 to 6 times as long as expected.

Not all computers are exactly as efficient as expected on a given calculation. Therefore, the Duration Correction Factor was developed to deal with these discrepencies. Some of the things that matter that are not measured in the benchmarks are: L1 and L2 cache sizes and speeds, memory bandwidth, memory size (too small can cause thrashing and very slow results - see BURP and Render@Home for worst case examples). If all of the DCFs for the project average about 1, then this is the best that can be expected.

The real problems arise when a project is off by more than an order of magnitude in its estimates. If the actual averate is around 5, it is not too bad, but it could be improved.

Hi John,

Do you mean that if we would increase the FP estimate for our workunit, the estimate made by the boinc client would be better? (that would be easy to do)

Thanks

Andre

It seems that you picked a system that is a bit more efficient than average at these computations. Of the machines that I have attached, most are in the range of 5 to 6 which means that the calculations are taking 5 to 6 times as long as expected.

Not all computers are exactly as efficient as expected on a given calculation. Therefore, the Duration Correction Factor was developed to deal with these discrepencies. Some of the things that matter that are not measured in the benchmarks are: L1 and L2 cache sizes and speeds, memory bandwidth, memory size (too small can cause thrashing and very slow results - see BURP and Render@Home for worst case examples). If all of the DCFs for the project average about 1, then this is the best that can be expected.

The real problems arise when a project is off by more than an order of magnitude in its estimates. If the actual averate is around 5, it is not too bad, but it could be improved.

Will completed WUs that finish past the deadline still count or should I just abort those? I also had received too many for me to complete even with my machine running 24/7. I don't want to waste time if it doesn't matter. Thanks.
____________

Will completed WUs that finish past the deadline still count or should I just abort those? I also had received too many for me to complete even with my machine running 24/7. I don't want to waste time if it doesn't matter. Thanks.

10-4 I just didn't want to waste the time if the result didn't matter.

Thanks.
____________

Will completed WUs that finish past the deadline still count or should I just abort those? I also had received too many for me to complete even with my machine running 24/7. I don't want to waste time if it doesn't matter. Thanks.

I fear no unfortunately. If the deadline past, the server produce and distribute a new result, regarding the late result as "no reply".

Will completed WUs that finish past the deadline still count or should I just abort those? I also had received too many for me to complete even with my machine running 24/7. I don't want to waste time if it doesn't matter. Thanks.

I fear no unfortunately. If the deadline past, the server produce and distribute a new result, regarding the late result as "no reply".

Actually, it depends on how late the return is, how fast the server is at generating new tasks when a result is late, how fast they are handed out, and how fast they are returned.

If the late work is returned before the replacement is actually sent, the replacement will not be sent. If the late work is returned before the replacement is verified, then the late work will at least contribute towards the verification.

If the late work is returned after a quorum is met and verified, it counts for nothing.

We calculate the estimate based on the runtime of a result on a 3 GHz linux box. I assume that when the estimated runtime on my boinc manager is equal to the actual runtime, the FP estimate must be correct and will correctly scale to all other platforms. If that is not the case, could there be a problem in boinc?

BOINC is certainly partly to blame - it asked for two days worth (1 day per logical CPU) when it should have asked for about 7 hours of work (20% of 18 hours a day available on each logical CPU).

Thanks for the info Yoda.

I've increased the flops estimate to almost double to what it was before. Please let us know if that makes the estimate on your client better. This will always be a problematic point though since the performance on linux, windows and macs for Charmm is not the same (and won't be any time soon), but boinc only allows one estimate to be provided for a workunit (I think). We have to see how we can tackle that problem.

Thanks
Andre

PS It might take a little while until the workunits with the new estimated flops are distributed.

This is still a problem. I just re-attached my 3.4GHz Pentium 4 with HT, running Windows.

It typically takes 10 hours per work unit and BOINC asked for 1 day worth. Your server sent 19 work units, which will take this computer 4 full days if I suspend everything else on it and don't switch it off at night.

Docking has only a 20% share of resources and the computer is only running 18 hours a day so I only needed 1 work unit. BOINC is certainly partly to blame - it asked for two days worth (1 day per logical CPU) when it should have asked for about 7 hours of work (20% of 18 hours a day available on each logical CPU).

But the WU time estimates are still way off the mark. Had BOINC requested only 7 hours worth, your scheduler (which seems to think they will take 1:45 each when in reality they take 10 hours) might still have sent 4 work units (about 40 hours of crunching, rather than the 7 needed).

The fastest I've completed a WU that was valid on my Athlon XP 3000+ was 4 hours (running Linux) and 6.5 hours (running Windows).

Even if we take into account that these WU take longer under Windows than Linux, it's still too big a discrepancy. My fastest PC (Athlon 64 at 2.6GHz with 500MHz dual channel DDR) still takes over 5 hours per work unit under Windows...

Just for the record, I'm running with the stock BOINC client and low benchmarks, so the discrepancy in estimated time to complete is not a matter of inflated benchmarks.

Might be worth comparing the benchmarks on your 3GHz Linux system and my 3.4Ghz Windows system?

The worst problem only exhibits itself until the client has completed its first result. After that point the Duration Correction Factor will be large enough to prevent download of vast quantities of work.

I think over-estimating the time taken to complete a work unit is less likely to cause problems than under-estimating. Doubling the estimate is certainly a step in the right direction.

When it's under-estimated, many (windows) hosts will download more work than they can handle. When it is over-estimated, linux hosts will not downoad enough, but there is less chance of work units going over their deadline.

The DCF will adjust, sure. But it's too late if the computer has too much to handle in the first place due to the under-estimated time.

On the other hand, with Linux hosts computing them quicker, their DCF will also adjust quicker, so the impact of the inaccurate time estimate is not as great.

I hope I'm making sense?

With regard to the BIONC scheduler being fixed in new versions (as well as Linux benchmarks), that's great, especially for people who run more than one project I look forward to it :D

FWIW, I have just loaded BOINC on the P4/3.4 (with Ubuntu 6.10 Live CD) and am running a Docking WU to compare times. Hope it has enough RAM as I haven't got a partition to put Linux on with this box (using a RAMdisk). Fingers crossed.

EDIT: Well, that didn't work either. Even though ulimit is unlimited on this distro, it still crashed both WU with an 0x1 error. Back to Windows for me - it may be slow but it works.
____________


Join the #1 Aussie Alliance on Docking@Home

Hello Webmaster Yoda, I found this information by Cold Shot in another thread, that may help your Linux problem with Ubuntu 6.10,

Ubuntu 6.10
Linux 2.6.17-10
Boinc client and manager 5.4.11-1

This distro has no run_manager or run_client
Added ulimit -s unlimited to boinc-client a script file located in /etc/init.d

This seems to have fixed the problem. I'm running Linux on a slow box (Celeron 1200 MHz, 384 mem). It took me a little over 8.5 hours to complete the first work unit.

Hope this helps.
____________

Hello Webmaster Yoda, I found this information by Cold Shot in another thread, that may help your Linux problem with Ubuntu 6.10,

Ubuntu 6.10
Linux 2.6.17-10
Boinc client and manager 5.4.11-1

This distro has no run_manager or run_client
Added ulimit -s unlimited to boinc-client a script file located in /etc/init.d

This seems to have fixed the problem. I'm running Linux on a slow box (Celeron 1200 MHz, 384 mem). It took me a little over 8.5 hours to complete the first work unit.

Hope this helps.

Hello Webmaster Yoda.

I use Redhat distributions, but I've heard it mentioned in the forums that the Debian maintainers have a package that installs the BOINC client as a service. I believe it's also available for Ubuntu.

I think someone got Linux/BOINC to work on the free VMware player with windows as the host.

I suspect that part of the problem you're experiencing is the big debug file that the Docking@Home writes. If the work unit succeeds, I think this file is replaced by a file that's only 8 or 9 bytes before it's uploaded. Until then, it grows rapidly and is probably using up your ram disk.

I'm on dial-up or I'd have tried the VMware approach myself. I suspect that the delayed writes that Linux uses are part of the reason it's faster. If that's the case, BOINC on Linux on VMware on Windows might even be faster than BOINC on Windows. There might also be different compilers used for the Linux version. I'm not sure on that. If you have a fast connection and some space free on your windows filesystem, this might be worth trying. I think it just creates files in the windows filesystem to hold the virtual Linux disk partitions, so you don't have to have unpartitioned space on your disk drive to create real Linux partitions.

I hope this helps,

-- David

Thanks David, I tried it under VMWare as well, but it (and Linux) dosn't like my wireless network, so I gave up on that idea.

It's summer here and with he hard-disk constantly on the go (particularly when running two docking work units side by side), it gets too hot and the computer crashes (and the work unit errors out in the process). I'll probably have to abandon Docking for the time being as the constant disk writes are wreaking havoc with my systems.

I don't get these lock-ups when running other BOINC projects.


____________


Join the #1 Aussie Alliance on Docking@Home

Thanks David, I tried it under VMWare as well, but it (and Linux) dosn't like my wireless network, so I gave up on that idea.

It's summer here and with he hard-disk constantly on the go (particularly when running two docking work units side by side), it gets too hot and the computer crashes (and the work unit errors out in the process). I'll probably have to abandon Docking for the time being as the constant disk writes are wreaking havoc with my systems.

The checkpointing method and period will be the first thing we work on when Charmm c33b1 is released by the Charmm developers. This hopefully won't take too long anymore and will bring the disk activity down to an acceptable level (although I'm kind of getting desperate slowly slowly...).

Also, we currently write a lot of debug information to the charmm logfile, which is also a cause of lots of disk activity; since we are in alpha we will have to do this to find out problems more easily. as soon as we have most of our pressing problems solved, we can cut back on this too.

Hello Webmaster Yoda.

I use Redhat distributions, but I've heard it mentioned in the forums that the Debian maintainers have a package that installs the BOINC client as a service. I believe it's also available for Ubuntu.

I think someone got Linux/BOINC to work on the free VMware player with windows as the host.

I suspect that part of the problem you're experiencing is the big debug file that the Docking@Home writes. If the work unit succeeds, I think this file is replaced by a file that's only 8 or 9 bytes before it's uploaded. Until then, it grows rapidly and is probably using up your ram disk.

I'm on dial-up or I'd have tried the VMware approach myself. I suspect that the delayed writes that Linux uses are part of the reason it's faster. If that's the case, BOINC on Linux on VMware on Windows might even be faster than BOINC on Windows. There might also be different compilers used for the Linux version. I'm not sure on that. If you have a fast connection and some space free on your windows filesystem, this might be worth trying. I think it just creates files in the windows filesystem to hold the virtual Linux disk partitions, so you don't have to have unpartitioned space on your disk drive to create real Linux partitions.

I hope this helps,

-- David