Double sized WUs are out: *variant*_long* and include a bonus of the 50% on the credits.

Double sized WUs are out: *variant*_long* and include a bonus of the 50% on the credits.

First one of these is completed in 7h32m (27116.891s).
Time per step (avg over 2500000 steps): 10.847ms
Claimed Credit: 23878 (that's increased 50% compared to a *_IBUCH_?_pYEEI_long_* WU)
Granted Credit: 35817 (standard 50% fast return bonus)
It gives 1.32 credit/sec. that's not bad at all :) My fastest GIANNI_DHFR1000 gives 1.5 credit/sec.
GPU usage is 62-64% on GTX 580, and 62-67% on GTX 480.

10-IBUCH_8_variantP_long-0-2-RND0787_0
Workunit 2166428
Aangemaakt 14 Dec 2010 19:08:59 UTC
Sent 14 Dec 2010 22:55:38 UTC
Received 15 Dec 2010 9:05:29 UTC
Server state Over
Outcome Success
Client state Geen
Exit status 0 (0x0)
Computer ID 35174
Report deadline 19 Dec 2010 22:55:38 UTC
Run time 35497.448383
CPU time 35436.44
stderr out <core_client_version>6.10.58</core_client_version>
<![CDATA[
<stderr_txt>
# Using device 0
# There is 1 device supporting CUDA
# Device 0: "GeForce GTX 480"
# Clock rate: 1.40 GHz
# Total amount of global memory: 1610153984 bytes
# Number of multiprocessors: 15
# Number of cores: 120
SWAN: Using synchronization method 0
MDIO ERROR: cannot open file "restart.coor"
# Time per step (avg over 2500000 steps): 14.200 ms
# Approximate elapsed time for entire WU: 35500.370 s
called boinc_finish

</stderr_txt>
]]>


Validate state Geldig
Claimed credit 23878.0787037037
Granted credit 35817.1180555555
application version ACEMD2: GPU molecular dynamics v6.13 (cuda31)

--------------------------------------------------------------------------------
14 Dec 2010 22:55:38 UTC 15 Dec 2010 9:05:29 UTC Completed and validated 35,497.45 35,436.44 23,878.08 35,817.12 ACEMD2: GPU molecular dynamics v6.13 (cuda31

--------------------------------------------------------------------------------
My first long WU!

Give me more please!
____________
Ton (ftpd) Netherlands

Well,

It seems that long WUs aren't afterall that much gain for everybody. We have dropped by 1000 WUs in progress in one single day. We may be trying to push the computations too far here. The last thing we want is to scare people out and run away from GPUGRID.

We are going to reconsider the strategy for "fast-track" WUs.

What do you think?

cheers,
ignasi

Ignasi,

If i had to choose: very long WU's or small quick WU's, but then a greater amount of WU's.

I am NOT scared, but give some cards (gts250) also something to crunch.
____________
Ton (ftpd) Netherlands

It seems that long WUs aren't afterall that much gain for everybody. We have dropped by 1000 WUs in progress in one single day. We may be trying to push the computations too far here. The last thing we want is to scare people out and run away from GPUGRID.

Sure.

I am submitting non-long WUs at the moment.

There's plenty of work for everybody always.

i

Well,

It seems that long WUs aren't afterall that much gain for everybody. We have dropped by 1000 WUs in progress in one single day. We may be trying to push the computations too far here. The last thing we want is to scare people out and run away from GPUGRID.

We are going to reconsider the strategy for "fast-track" WUs.

What do you think?

cheers,
ignasi

...
Having said that, I've got IBUCH_*_variantP_long running on all four hosts at the moment - I'll comment on your other questions when they've finished, in 5 - 38 hours from now.

@skgiven
Correct.
It's a matter of few days as you say.

@Retvari Zoltan
I do my best, thank you.

The *long* WUs are not meant to be something regular at all. We always stated that.
The problem is that I have abused of them in the rush for getting key results back for publication. The solution for this issue is not just extending WU length. It has to be well thought out and degrees of freedom to adjust, pinpointed. Classification of WU by card comp. capacity is certainly an option.

Back to science,
i

I do my best, thank you.

The *long* WUs are not meant to be something regular at all. We always stated that.
The problem is that I have abused of them in the rush for getting key results back for publication.

The solution for this issue is not just extending WU length. It has to be well thought out and degrees of freedom to adjust, pinpointed. Classification of WU by card comp. capacity is certainly an option.

Ignasi,

My gts250 does NOT receive any downloads for 24 hrs now.

All Kashiv-wu are cancelling after several hours on this card.

Is that the reason?


____________
Ton (ftpd) Netherlands

I just cruised by to say I'm lovin' the new wu's. ~21 hours on GTX260 but only ~55% utilization of the card. ??? Nice credits too :)

Good job.
____________

The problem is that I have abused of them in the rush for getting key results back for publication. The solution for this issue is not just extending WU length. It has to be well thought out and degrees of freedom to adjust, pinpointed. Classification of WU by card comp. capacity is certainly an option.

Computer ID 47762
Report deadline 19 Dec 2010 23:11:00 UTC
Run time 63982.28125
CPU time 17871.09
stderr out <core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
- exit code 98 (0x62)
</message>
<stderr_txt>
# Using device 0
# There are 2 devices supporting CUDA
# Device 0: "GeForce GTX 295"
# Clock rate: 1.24 GHz
# Total amount of global memory: 939327488 bytes
# Number of multiprocessors: 30
# Number of cores: 240
# Device 1: "GeForce GTX 295"
# Clock rate: 1.24 GHz
# Total amount of global memory: 939196416 bytes
# Number of multiprocessors: 30
# Number of cores: 240
MDIO ERROR: cannot open file "restart.coor"
ERROR: file tclutil.cpp line 31: get_Dvec() element 0 (b)
called boinc_finish

</stderr_txt>
]]>


Validate state

This one cancelled after almost 18 hrs. Windows XP - gtx 295.

Next one, please
____________
Ton (ftpd) Netherlands

Computer ID 47762
Report deadline 19 Dec 2010 23:11:00 UTC
Run time 71377.171875
CPU time 18345.22
stderr out <core_client_version>6.10.58</core_client_version>
<![CDATA[
<stderr_txt>
# Using device 1
# There are 2 devices supporting CUDA
# Device 0: "GeForce GTX 295"
# Clock rate: 1.24 GHz
# Total amount of global memory: 939327488 bytes
# Number of multiprocessors: 30
# Number of cores: 240
# Device 1: "GeForce GTX 295"
# Clock rate: 1.24 GHz
# Total amount of global memory: 939196416 bytes
# Number of multiprocessors: 30
# Number of cores: 240
MDIO ERROR: cannot open file "restart.coor"
# Using device 1
# There are 2 devices supporting CUDA
# Device 0: "GeForce GTX 295"
# Clock rate: 1.24 GHz
# Total amount of global memory: 939327488 bytes
# Number of multiprocessors: 30
# Number of cores: 240
# Device 1: "GeForce GTX 295"
# Clock rate: 1.24 GHz
# Total amount of global memory: 939196416 bytes
# Number of multiprocessors: 30
# Number of cores: 240
# Time per step (avg over 65000 steps): 34.528 ms
# Approximate elapsed time for entire WU: 86320.913 s
called boinc_finish

</stderr_txt>
]]>


Validate state Geldig
Claimed credit 23878.0787037037
Granted credit 35817.1180555555
application version ACEMD2: GPU molecular dynamics v6.13 (cuda31)

--------------------------------------------------------------------------------


--------------------------------------------------------------------------------
This one is succesfull.

Next one is processing!
____________
Ton (ftpd) Netherlands

Ignasi,
is there a way to sort out the long wu's by an app_info.xml? If, for example, the 6.13 app is only for the shorter wu's and a (suggested) 6.14 app is for the longer wu's everyone can decide for himself what type he wants to crunch. And without app_info he gets whats available.
Alexander

No problems with the long
WU's , but when checking upon my XP64 host, cause computing times got
longer on 'the same WU' types.
GPUz reported, after I did put an 470 next to the 480, the 470 runs in PCI-E x2
mode and the 480 in 1x mode?! Which is ofcoarse slower, not 16x but about 1.5
times.
OK, ( a bit) off topic, but very odd and haven't found a setting(BIOS) or reason,
as to why it does this. Looks like it doesn't like NVIDIA cards!

Have 2 ASUS P5E Mobo's, 1 with a Q6600 and ATI's HD4850 & HD5870, both running
PCI-E x16, ver2.0
The other has an X9650 (@3.5GHz)and a GTX480 and a 470, only difference is OS type.
When I try to force, in BIOS, the cards in 16x mode, it suddenly stops, no warning or fault, just 'hangs'.
Maybe the X38 chipset, as all the cards are from ASUS as well.

But I do see an unusual amount of computation errors on 200 series, also on FX
(Quadro) cards. (Too much difference in architecture between 200 and 400, 500
series?)





____________

Knight Who Says Ni N!

Presently, tasks are allocated acording to what Cuda capability your card has, which in turn is determined by your driver:

Have we run out of *long* wu's? I was getting one about 1 in 4 but now they have stopped or has something else changed? I like them :)
____________

Hmmmm. task 3445649

Run time 162790 seconds, CPU time 42746 seconds, is an awful lot of resources to throw at a "SWAN: FATAL : swanBindToTexture1D failed -- texture not found"

Any light to throw on this new error message yet? It's not been reported in the last 12 months, except by me - and that was on a completely different host.

Have we run out of *long* wu's? I was getting one about 1 in 4 but now they have stopped or has something else changed? I like them :)

Another of my hosts has just completed WU 2166545 - p19-IBUCH_10_variantP_long-0-2-RND8229.

It took 48.5 hours, so the forgone 'quick return' bonus cancelled out the 50% 'long' bonus. But, more importantly - and contrary to some complaints here - no second copy was created, and the work I did was 100% useful for the science.

I would be more worried about the "No heartbeat from core client for 30 sec - exiting" message.

A second task is not sent out until 48h after the initial task is sent (normally), so as yours returned in 48.5h then I guess the server just did not get round to issuing a resend before your task returned. Your task would have been fully used even if another task had been issued, as your task would be the first back, and if a task was sent out again, in most cases it would not start automatically, so if you return slightly later than 48h your task is still the most useful; un-started resends can be recalled. However, after 3 or 4days this is no longer the case; resends would have completed. Sometimes however the resends fail, but not too often, as they tend to go to reliable and faster cards. Begs the question why this allocation method is not initially used to determine long task hosts.

I've received two reissued IBUCH_*_variantP_long_ WUs.
Workunit 2166526
Workunit 2166431
I think these long WUs are waiting to be reissued, that's why we didn't receive much of them in the past 48 hours.

A second task is not sent out until 48h after the initial task is sent (normally), so as yours returned in 48.5h then I guess the server just did not get round to issuing a resend before your task returned. Your task would have been fully used even if another task had been issued, as your task would be the first back, and if a task was sent out again, in most cases it would not start automatically, so if you return slightly later than 48h your task is still the most useful; un-started resends can be recalled. However, after 3 or 4days this is no longer the case; resends would have completed. Sometimes however the resends fail, but not too often, as they tend to go to reliable and faster cards. Begs the question why this allocation method is not initially used to determine long task hosts.

We know all this. My comment was primarily aimed at Saenger, who seemed to be under the impression that a new task would be created, allocated, downloaded, and run unconditionally at 48 hours and 1 second - thus wasting electricity and CPU cycles on the second host. I thought a counter-example might help to set his mind at rest.

http://www.gpugrid.net/workunit.php?wuid=2166431
A fine example of the reason not to send these to CC1.1 cards.

The resend could be sent after 20min or 5h, but if it goes to a fast card, it will be back fairly quickly. Very few tasks returned after 4days would be worth anything, but after 2days and a few hours their value is high for the project.

...and the other one I've mentioned (Workunit 2166526) is a fine example of the reason not to send these to hosts with low RAC.

I think TDCF is likely to be a most unhelpful measure of past performance - not least because I have a suspicion that the current duration estimates (as defined by <rsc_fpops_est>) don't adequately reflect the work complexity of different task types, and the efficiency of processing of the ones, like GIANNI, which take advantage of the extra processing routines in the newer applications.

And if the estimates vary, then the correction factors will vary too. TDCF will merely reflect the initial fpops_est error of the most recently run tasks.

s0r78-TONI_HIVMSMWO1-0-6-RND2382_1: 9,930.70 seconds
p32-IBUCH_1_pYEEI_long_101130-8-10-RND3283_1: 45,855.31 seconds

Same host, same card. Both jobs were issued with the same
<rsc_fpops_est>1000000000000000.000000</rsc_fpops_est>

TDCF will never be able to cope fully with an almost five-fold variation between consecutive tasks.

Hmmmm. task 3445649

Run time 162790 seconds, CPU time 42746 seconds, is an awful lot of resources to throw at a "SWAN: FATAL : swanBindToTexture1D failed -- texture not found"

Any light to throw on this new error message yet? It's not been reported in the last 12 months, except by me - and that was on a completely different host.

It seems that long WUs aren't afterall that much gain for everybody. We have dropped by 1000 WUs in progress in one single day. We may be trying to push the computations too far here. The last thing we want is to scare people out and run away from GPUGRID.

You may have to consider external factors, as well as your own internal choices.

Do you have a medium/long term record of that "WUs in progress" figure? I suspect that you may have been affected by that other big NVidia beast in the BOINC jungle - SETI@home.

They have been effectively out of action since the end of October. You may well have been benefitting from extra resources during that time. SETI has been (slowly and intermittently) getting back up to speed over the last week or so, and as they do so, you will inevitably lose some volunteers (or some share of their machines).

Having said that, I've got IBUCH_*_variantP_long running on all four hosts at the moment - I'll comment on your other questions when they've finished, in 5 - 38 hours from now.

Well,

It seems that long WUs aren't afterall that much gain for everybody. We have dropped by 1000 WUs in progress in one single day. We may be trying to push the computations too far here. The last thing we want is to scare people out and run away from GPUGRID.

We are going to reconsider the strategy for "fast-track" WUs.

What do you think?

cheers,
ignasi