To Linux crunchers: lower credits issue


Advanced search

Message boards : Number crunching : To Linux crunchers: lower credits issue

Sort
Author Message
Profile suguruhirahara
Forum moderator
Volunteer tester
Avatar

Joined: Sep 13 06
Posts: 282
ID: 15
Credit: 56,614
RAC: 0
Message 1467 - Posted 17 Nov 2006 9:19:32 UTC
Last modified: 19 Nov 2006 10:29:05 UTC

Hello,

As many of you have already noticed, in this project a linux machine cannot be granted credits as much as Windows can. Compared with the amount of credits which Windows usually claims, approximately only 40% of that is claimed from a linux machine, according to my calculation with this (on Windows) and this (on Linux). Probably some of you are frustrated with this point.

However, please keep in mind that this issue is due to not the team of this project, but the BOINC framework of credit system.

As I noticed on several threads of this project, there is a thread on World Community Grid, which explains this point (disclaimer: I'm not sure whether this is genuine information, since I've never contacted with Dr David Anderson).

The first post on the thread says that to solve the issue "Linux earns fewer points per workunit then the other platforms", "they will be compiling the Linux agent in the 5.8 release with optimizations that are comparable to the optimizations used on the other platforms."

This results in that with the release of BOINC version 5.8 linux hosts can be granted the fair amount of credits compared with Windows (or Mac).

The new version will be, according to the post, available in a month. Since I assume that the team has other tasks which are prior to the improving of credit system, please wait for the moment to have a chance for being granted the fair amount of credits.

Actually I'm not a member of the team, but I post this message, because, as well as the team, I'd love you to know the fact around the issue.

(To the team: please correct the message if there is anything I'm missing or wrong.)

Thanks for reading and understanding,
suguruhirahara
____________

I'm a volunteer participant; my views are not necessarily those of Docking@Home or its participating institutions.

Profile [B^S] Doug Worrall
Volunteer tester
Avatar

Joined: Sep 13 06
Posts: 127
ID: 74
Credit: 11,046
RAC: 0
Message 1468 - Posted 17 Nov 2006 11:03:45 UTC


Hello, suguruhirahara

I believe, you got all the information, in this Post, too help Linux users.
Good to know that the credit system will be addressed soon.From day 1, have tried
to the best of my abbility, too keep crunching.Folding experiments, are my 1st.
choice of experiments. The Scientists and personel should be doing Science
etc..,while suguruhirahara
Forum moderator, is doing a bang up job, communicating issues that are important,
freeing up time for the Docking "personel" too upgrade apps, etc..

Thanks Docking Moderators, personel, Scietists

Sluger

____________

Profile Andre Kerstens
Forum moderator
Project tester
Volunteer tester
Avatar

Joined: Sep 11 06
Posts: 749
ID: 1
Credit: 15,199
RAC: 0
Message 1471 - Posted 18 Nov 2006 1:10:03 UTC - in response to Message ID 1467 .

Thanks Suguru!
This is a great explanation :-)
Andre

Hello,

As many of you have already noticed, in this project a linux machine cannot be granted credits as much as Windows can. Compared with the amount of credits which Windows usually claims, approximately only 40% of that is claimed from a linux machine, according to my calculation with this (on Windows) and this (on Linux). Probably some of you are frustrated with this point.

However, please keep in mind that this issue is due to not the team of this project, but the BOINC framework of credit system.

As I noticed on several threads of this project, there is a thread on World Community Grid, which explains this point (disclaimer: I'm not sure whether this is genuine information, since I've never contacted with Dr David Anderson).

The first post on the thread says that to solve the issue "Linux earns fewer points per workunit then the other platforms", "they will be compiling the Linux agent in the 5.8 release with optimizations that are comparable to the optimizations used on the other platforms."

This results in that with the release of BOINC version 5.8 linux hosts can be granted the fair amount of credits compared with Windows (or Mac).

The new version will be, according to the post, available in a month. Since I assume that the team has other tasks which are prior to the improving of credit system, please wait for the moment to have a chance for being granted the fair amount of credits.

Actually I'm not a member of the team, but I post this message, because, as well as the team, I'd love you to know the fact around the issue.

(To the team: please correct the message if there is anything I'm missing or wrong.)

Thanks for reading and understanding,
suguruhirahara


____________
D@H the greatest project in the world... a while from now!
Profile suguruhirahara
Forum moderator
Volunteer tester
Avatar

Joined: Sep 13 06
Posts: 282
ID: 15
Credit: 56,614
RAC: 0
Message 1517 - Posted 20 Nov 2006 11:23:45 UTC
Last modified: 1 Dec 2006 2:46:34 UTC

Hi all:)

If you're interested in how discussion could be advanced so fast once the theme is related with credits, read this thread.

http://www.worldcommunitygrid.org/forums/wcg/viewthread?thread=9549

Since I'm of course a volunteer and I cannot spare time in moderating forums so much, and not a native English speaker, I couldn't manage forums if they would be overheated. To prevent us from getting the such a situation, IMO until this project go public there should be a kind of consensus that most of us can agree with. We should tackle this issue near in future, at least after the new client of linux will be released.

Regarding this project I'd love you to make sure what I mentioned at the first post of this thread.

Thanks for reading and understanding,
suguruhirahara

Edit: just correcting several words...
____________

I'm a volunteer participant; my views are not necessarily those of Docking@Home or its participating institutions.

Profile Conan
Volunteer tester
Avatar

Joined: Sep 13 06
Posts: 219
ID: 100
Credit: 4,256,493
RAC: 0
Message 1677 - Posted 1 Dec 2006 20:15:02 UTC

@suguruhirahara, your calculation of 40% difference between Windows and Linux claims seems a bit high.
The difference between from what I have seen of other computers in my Linux pool is closer to 25%-30%, with Windows machines claiming 80+ and Linux claiming 20-25 on average.
The granted credit percent can be sometimes lower still.
It would be a good idea when the project moves away from Boinc benchmarks to do credit balance and award all the Linux machines (possibly the Macs too?) a credit boost to bring tham back level with Windows. After all they have cruched the same amount of work (and much faster) but are getting a quarter of the credit.
____________

Profile suguruhirahara
Forum moderator
Volunteer tester
Avatar

Joined: Sep 13 06
Posts: 282
ID: 15
Credit: 56,614
RAC: 0
Message 1760 - Posted 13 Dec 2006 12:17:11 UTC
Last modified: 13 Dec 2006 12:17:49 UTC

The new version (5.6.4 to 5.7.5) of BOINC for linux to be tested became available on the download page . It implemented new feature of 'advanced GUI only', but as far as I checked the benchmark system was improved a little though it's not perfect. Try it if you want to.

suguruhirahara
____________

I'm a volunteer participant; my views are not necessarily those of Docking@Home or its participating institutions.

Profile Webmaster Yoda
Volunteer tester
Avatar

Joined: Nov 14 06
Posts: 43
ID: 271
Credit: 6,498
RAC: 0
Message 1768 - Posted 14 Dec 2006 14:46:26 UTC - in response to Message ID 1677 .
Last modified: 14 Dec 2006 14:49:14 UTC

@suguruhirahara, your calculation of 40% difference between Windows and Linux claims seems a bit high.
The difference between from what I have seen of other computers in my Linux pool is closer to 25%-30%, with Windows machines claiming 80+ and Linux claiming 20-25 on average.


As you'd know, CPU time is one factor in the way BOINC claims credit. We also know the app is much faster under Linux.

Example 1: My Athlon XP 3000+
Windows: approx 23,500 seconds for 83 credits (12.7 per hour)
Linux: approx. 14,000 seconds for 28 credits (7.2 per hour)

Example 2: My Pentium 4/3.4 with HT
Windows: approx 35,500 seconds for 86 credits (8.7 per hour per thread)
Linux: approx. 20,000 seconds for 29 credits (5.2 per hour per thread)

The total claimed per work unit under Linux is about 33% of the claim under Windows. But it takes about 58% of CPU time. On this project at least, per hour of CPU time, the Linux host claims about 55-60% of the Windows host, not 25-30%.

Still a big difference (and a BOINC problem), but not as bad as when you look at it purely from a credit per work unit perspective.
____________


Join the #1 Aussie Alliance on Docking@Home
Rene
Volunteer tester
Avatar

Joined: Oct 2 06
Posts: 121
ID: 160
Credit: 109,415
RAC: 0
Message 1775 - Posted 15 Dec 2006 6:18:13 UTC - in response to Message ID 1760 .
Last modified: 15 Dec 2006 6:24:57 UTC

The new version (5.6.4 to 5.7.5) of BOINC for linux to be tested became available on the download page . It implemented new feature of 'advanced GUI only', but as far as I checked the benchmark system was improved a little though it's not perfect. Try it if you want to.

suguruhirahara


Benchmark indeed looks higher...

5.7.5:
Measured floating point speed 1301.74 million ops/sec
Measured integer speed 2277.85 million ops/sec

5.4.9:
Measured floating point speed 993.78 million ops/sec
Measured integer speed 1740.58 million ops/sec

Result... my linux host is now claiming approx 29-30 credits/13.400 seconds.
(5.4.9 approx 22-23/13.400)
As far as I can tell this is close to bench-vs-claim of my Windows host.

Besides from that the 5.7.5 GUI looks a lot better... ;-)



____________
Profile suguruhirahara
Forum moderator
Volunteer tester
Avatar

Joined: Sep 13 06
Posts: 282
ID: 15
Credit: 56,614
RAC: 0
Message 1776 - Posted 15 Dec 2006 7:52:46 UTC - in response to Message ID 1775 .

Besides from that the 5.7.5 GUI looks a lot better... ;-)

Yes, this is one of the key update features. Boinc manager 5.4.9 is frankly not...good one. The new one is quite better:)

____________

I'm a volunteer participant; my views are not necessarily those of Docking@Home or its participating institutions.
[B^S] Morgan the Gold
Volunteer tester
Avatar

Joined: Oct 2 06
Posts: 41
ID: 170
Credit: 138,735
RAC: 0
Message 1880 - Posted 25 Dec 2006 2:58:41 UTC

5.7.5 linux is closer to windoze for benchmark but hard to believe A & B are the same machine. with 800 less mflops &300 less mips linux still appears to be over 20% faster
____________

Profile David Ball
Forum moderator
Volunteer tester
Avatar

Joined: Sep 18 06
Posts: 274
ID: 115
Credit: 1,634,401
RAC: 0
Message 1887 - Posted 25 Dec 2006 19:52:33 UTC - in response to Message ID 1880 .

5.7.5 linux is closer to windoze for benchmark but hard to believe A & B are the same machine. with 800 less mflops &300 less mips linux still appears to be over 20% faster


I think that has something to do with the D@H project client writing to disk so much. Linux seems to handle that better by buffering and delaying the actual writes longer. IIRC, someone actually took a Windows XP machine, loaded VMware on it, and found that a WU ran faster in Linux on VMware on WinXP, than it ran natively on just WinXP on the same machine.

-- David
Profile Webmaster Yoda
Volunteer tester
Avatar

Joined: Nov 14 06
Posts: 43
ID: 271
Credit: 6,498
RAC: 0
Message 1893 - Posted 29 Dec 2006 4:00:15 UTC - in response to Message ID 1887 .

IIRC, someone actually took a Windows XP machine, loaded VMware on it, and found that a WU ran faster in Linux on VMware on WinXP, than it ran natively on just WinXP on the same machine.


My 3.4GHz Pentium 4 with HT took 5.5 hours for a single WU in Linux under VMWare (while also running BOINC in the host O/S). Running Docking in the host O/S (Windows 2003 SBS), work units took on average about 10 hours.

Whether this is related to disk performance, I don't know - it was using the same hard-drive and (presumably) still had to patch it through the host O/S.

Profile David Ball
Forum moderator
Volunteer tester
Avatar

Joined: Sep 18 06
Posts: 274
ID: 115
Credit: 1,634,401
RAC: 0
Message 1895 - Posted 29 Dec 2006 16:23:03 UTC - in response to Message ID 1893 .

IIRC, someone actually took a Windows XP machine, loaded VMware on it, and found that a WU ran faster in Linux on VMware on WinXP, than it ran natively on just WinXP on the same machine.


My 3.4GHz Pentium 4 with HT took 5.5 hours for a single WU in Linux under VMWare (while also running BOINC in the host O/S). Running Docking in the host O/S (Windows 2003 SBS), work units took on average about 10 hours.

Whether this is related to disk performance, I don't know - it was using the same hard-drive and (presumably) still had to patch it through the host O/S.


I think that Linux just stuffs the data the application writes into a buffer and returns success to the program that did the writing. Then Linux has a kernel thread which does the actual writes to disk about every 5 seconds. This lets the actual disk writes happen in parallel with the application rather than the application waiting for each of them to complete. I'm not sure of the exact windows algorithm but it seems like windows not only does the write while making the application wait for the write to complete, but charges that time to the application.

Even if VMware goes through Windows for the writes by the VM Linux kernel thread, it must be done by a different VMware Windows thread and occur in parallel with the Linux application running.

-- David

Profile Trog Dog
Volunteer tester
Avatar

Joined: Nov 14 06
Posts: 16
ID: 279
Credit: 145,805
RAC: 0
Message 1902 - Posted 29 Dec 2006 22:32:49 UTC - in response to Message ID 1893 .

IIRC, someone actually took a Windows XP machine, loaded VMware on it, and found that a WU ran faster in Linux on VMware on WinXP, than it ran natively on just WinXP on the same machine.


My 3.4GHz Pentium 4 with HT took 5.5 hours for a single WU in Linux under VMWare (while also running BOINC in the host O/S). Running Docking in the host O/S (Windows 2003 SBS), work units took on average about 10 hours.

Whether this is related to disk performance, I don't know - it was using the same hard-drive and (presumably) still had to patch it through the host O/S.



Is the vmware actual wallclock time or reported cpu time? I'm pretty sure that this was looked at on another project and it was found that the cputime reported under vmware was substantially different even when "nothing else" was running on vmware or the host system.
Profile Webmaster Yoda
Volunteer tester
Avatar

Joined: Nov 14 06
Posts: 43
ID: 271
Credit: 6,498
RAC: 0
Message 1903 - Posted 30 Dec 2006 7:40:45 UTC - in response to Message ID 1902 .
Last modified: 30 Dec 2006 7:41:10 UTC

Is the vmware actual wallclock time or reported cpu time?


I'm talking about reported CPU time in both cases. No idea how long it took in real time.
Profile Webmaster Yoda
Volunteer tester
Avatar

Joined: Nov 14 06
Posts: 43
ID: 271
Credit: 6,498
RAC: 0
Message 1973 - Posted 7 Jan 2007 4:34:42 UTC
Last modified: 7 Jan 2007 4:37:09 UTC

All this talk about newer BOINC versions fixing the disparity between Linux and Windows got me curious. I downloaded BOINC 5.8.1 (development version) to my 3.4GHz Pentium 4 (HT) which is dual boot now and I ran benchmarks a few times, with mixed results.

With BOINC set to use only 1 logical processor, the Linux client was close to the Windows client - about 5% lower in Linux. With BOINC using both processors, Linux was still left out in the cold.

Using 1 processor
Windows XP: 1687 FP and 2428 Int. = 8.57 CPH (Credits Per Hour)
Ubuntu 6.10: 1262 FP and 2657 Int. = 8.16 CPH

Using 2 processors
Windows XP: 1453 FP and 1783 int = 13.48 CPH (two threads at 6.74 each)
Ubuntu 6.10: 949 FP and 1519 Int = 10.28 CPH (two threads at 5.14 each)

Of course, it won't matter here with fixed credits coming soon, but I thought I'd report my findings anyway as it will still matter on many other projects.
____________


Join the #1 Aussie Alliance on Docking@Home

(retired account)
Volunteer tester

Joined: Nov 22 06
Posts: 62
ID: 331
Credit: 158,686
RAC: 0
Message 2025 - Posted 10 Jan 2007 10:26:26 UTC - in response to Message ID 1973 .
Last modified: 10 Jan 2007 10:28:31 UTC

I also upgraded my Dual Athlon XP 2 GHz running Ubuntu 6.10 to the new BOINC version 5.8.1 (development version) now and the claimed credits have increased significantly:

Previous BOINC 5.4.9
per cpu: 1049 FP and 1817 Int. = 6.46 claimed credits per hour
total: 2098 FP and 3634 Int. = 12.92 claimed credits per hour

Current BOINC 5.8.1
per cpu: 1451 FP and 2555 int = 8.96 claimed credits per hour
total: 2902 FP and 5110 int = 17.92 claimed credits per hour

Regards

Alex


____________

Profile suguruhirahara
Forum moderator
Volunteer tester
Avatar

Joined: Sep 13 06
Posts: 282
ID: 15
Credit: 56,614
RAC: 0
Message 2330 - Posted 27 Jan 2007 8:17:30 UTC

Hello all,

Here's an information.
As subscribers for boinc_alpha mailing list have probably noticed, there was a introduction by an official developer of boinc, mr rom walton, in the thread "[boinc_alpha] Benchmarks: Linux vs Windows" that benchmark have been improved in the latest beta release. Thus I guess that in the next release of stable version the issue will be fixed finally. Actually this project is going to adopt an original way of granting credits, so it won't relate to the issue.

Thanks for reading,
suguruhirahara
____________

I'm a volunteer participant; my views are not necessarily those of Docking@Home or its participating institutions.

Message boards : Number crunching : To Linux crunchers: lower credits issue

Database Error
: The MySQL server is running with the --read-only option so it cannot execute this statement
array(3) {
  [0]=>
  array(7) {
    ["file"]=>
    string(47) "/boinc/projects/docking/html_v2/inc/db_conn.inc"
    ["line"]=>
    int(97)
    ["function"]=>
    string(8) "do_query"
    ["class"]=>
    string(6) "DbConn"
    ["object"]=>
    object(DbConn)#23 (2) {
      ["db_conn"]=>
      resource(108) of type (mysql link persistent)
      ["db_name"]=>
      string(7) "docking"
    }
    ["type"]=>
    string(2) "->"
    ["args"]=>
    array(1) {
      [0]=>
      &string(51) "update DBNAME.thread set views=views+1 where id=103"
    }
  }
  [1]=>
  array(7) {
    ["file"]=>
    string(48) "/boinc/projects/docking/html_v2/inc/forum_db.inc"
    ["line"]=>
    int(60)
    ["function"]=>
    string(6) "update"
    ["class"]=>
    string(6) "DbConn"
    ["object"]=>
    object(DbConn)#23 (2) {
      ["db_conn"]=>
      resource(108) of type (mysql link persistent)
      ["db_name"]=>
      string(7) "docking"
    }
    ["type"]=>
    string(2) "->"
    ["args"]=>
    array(3) {
      [0]=>
      object(BoincThread)#3 (16) {
        ["id"]=>
        string(3) "103"
        ["forum"]=>
        string(1) "2"
        ["owner"]=>
        string(2) "15"
        ["status"]=>
        string(1) "0"
        ["title"]=>
        string(39) "To Linux crunchers: lower credits issue"
        ["timestamp"]=>
        string(10) "1169885850"
        ["views"]=>
        string(4) "1911"
        ["replies"]=>
        string(2) "17"
        ["activity"]=>
        string(20) "1.4308522806097e-125"
        ["sufferers"]=>
        string(1) "0"
        ["score"]=>
        string(1) "0"
        ["votes"]=>
        string(1) "0"
        ["create_time"]=>
        string(10) "1163755172"
        ["hidden"]=>
        string(1) "0"
        ["sticky"]=>
        string(1) "0"
        ["locked"]=>
        string(1) "0"
      }
      [1]=>
      &string(6) "thread"
      [2]=>
      &string(13) "views=views+1"
    }
  }
  [2]=>
  array(7) {
    ["file"]=>
    string(63) "/boinc/projects/docking/html_v2/user/community/forum/thread.php"
    ["line"]=>
    int(184)
    ["function"]=>
    string(6) "update"
    ["class"]=>
    string(11) "BoincThread"
    ["object"]=>
    object(BoincThread)#3 (16) {
      ["id"]=>
      string(3) "103"
      ["forum"]=>
      string(1) "2"
      ["owner"]=>
      string(2) "15"
      ["status"]=>
      string(1) "0"
      ["title"]=>
      string(39) "To Linux crunchers: lower credits issue"
      ["timestamp"]=>
      string(10) "1169885850"
      ["views"]=>
      string(4) "1911"
      ["replies"]=>
      string(2) "17"
      ["activity"]=>
      string(20) "1.4308522806097e-125"
      ["sufferers"]=>
      string(1) "0"
      ["score"]=>
      string(1) "0"
      ["votes"]=>
      string(1) "0"
      ["create_time"]=>
      string(10) "1163755172"
      ["hidden"]=>
      string(1) "0"
      ["sticky"]=>
      string(1) "0"
      ["locked"]=>
      string(1) "0"
    }
    ["type"]=>
    string(2) "->"
    ["args"]=>
    array(1) {
      [0]=>
      &string(13) "views=views+1"
    }
  }
}
query: update docking.thread set views=views+1 where id=103