I have uploaded the new CUDA3.1 application for everyone.
Remember to use SWAN_SYNC=0 for maximum performance (at the price of one CPU core).

Let us know the performance improvement.

gdf

Anyone with a CUDA3.1 driver will receive it.
(Windows only for now)

gdf

Got 6.05 cuda tasks, what to do to recieve 3.1 tasks? Beta 3.1 run nice on my GTX260.

For point hunters ... make sure you don't cancel too many or the server will shut you off for a while ... possibly a long while ~ many hours!!!

I know that, but my 65nm GTX260 crashes any CUDA 2.2/2.3 task. It was usable a year ago and from that time I am crunching other project on it, but not GPUGRID. I load (and cancel) task time to time to not reach daily quota. Sometimes i recieve 3.1 beta task that rise daily quota.

GDF, are you sure that server should send standard tasks for non-Fermi class GPU with 257.21 drivers as CUDA 3.1?

My fault. The server was not giving out cuda3.1 jobs to anyone.
Now it should be fixed.

gdf

For Linux, when ?

Please.

The beta workunit has a size of the simulation box of 64x64x64. Cuda3.1 optimizes FFT in multiple of 2. So it was a bit faster and other machines (WinXP) reproduced it. As they were overclocked even improved the reference speed we have found locally.

After these tests in gpugrid, it seems that other workunits are not any better than cuda3.0. I will do some tests next week on other workunit types.

Nevertheless, CUDA3.1 works well also on holder card, and seems to fix several bug. So it is a good release.

gdf

@GDF Please read this and give me an optout of 3.1 if that's possible because the result will be the probable loss of two crunchers or more for this project.

This CUDA 3010 WU look promising!
http://www.gpugrid.net/result.php?resultid=2611239

It finished about 40% faster than the CUDA 3000 tasks. So although there are some that are about 50% slower, there are some faster WU's as well.

GDF just needs to work out what the difference is, and make sure we get the fast ones rather than the slow ones.

Good luck with that,

Details:

The fast one,
2611239 1660106 3 Jul 2010 13:39:37 UTC 3 Jul 2010 18:51:08 UTC Completed and validated 6,601.92 6,598.16 4,727.93 7,091.89 ACEMD2: GPU molecular dynamics v6.09 (cuda31)

Name I531-TONI_KIDln-13-100-RND2019_0
Workunit 1660106
Created 3 Jul 2010 13:19:57 UTC
Sent 3 Jul 2010 13:39:37 UTC
Received 3 Jul 2010 18:51:08 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x0)
Computer ID 71363
Report deadline 8 Jul 2010 13:39:37 UTC
Run time 6601.921875
CPU time 6598.156
stderr out

<core_client_version>6.10.56</core_client_version>
<![CDATA[
<stderr_txt>
# Using device 0
# There is 1 device supporting CUDA
# Device 0: "GeForce GTX 470"
# Clock rate: 1.41 GHz
# Total amount of global memory: 1341718528 bytes
# Number of multiprocessors: 14
# Number of cores: 112
SWAN: Using synchronization method 0
MDIO ERROR: cannot open file "restart.coor"
# Time per step (avg over 1300000 steps): 5.077 ms
# Approximate elapsed time for entire WU: 6599.953 s
called boinc_finish

</stderr_txt>
]]>

Validate state Valid
Claimed credit 4727.92939814815
Granted credit 7091.89409722222
application version ACEMD2: GPU molecular dynamics v6.09 (cuda31)

An example of the older CUDA 3000 tasks,
2603160 1654062 2 Jul 2010 1:29:51 UTC 2 Jul 2010 6:15:33 UTC Completed and validated 8,218.06 8,105.92 4,428.01 6,642.02 ACEMD2: GPU molecular dynamics v6.05 (cuda30)

An example of the newer slow WUs,
2611961 1657969 3 Jul 2010 17:01:16 UTC 3 Jul 2010 22:26:14 UTC Completed and validated 12,871.38 12,867.41 4,535.61 6,803.41 ACEMD2: GPU molecular dynamics v6.09 (cuda31)

These CUDA 3.1 I531-TONI_KIDln WU’s are clearly much faster than the CUDA 3000 WUs, and the 3.1 TONI_HERG WU's are just the same as 3.0, but the TONI_CAPBIND, KASHIF_HIVPR are much slower.

TONI_KID fast, and again

Before last weekend’s downtime my Fermi on XP had a steady RAC of 65K. It has now dropped to 55K and is showing no signs of increasing.
If I just crunched the TONI_CAPBIND it would drop to 45K!
If I just crunched KASHIF_HIVPR it would drop to about 53K
If I could just crunch TONI_KIDln it would rise to 92K. Yes, these are twice as fast as TONI_CAPBIND under CUDA 3.1. So, as you can see there is a massive performance difference between tasks!
Before you (GPUGrid team) sit down to work out why this is, could you build the Work Units according to this, so the Fermi’s just get TONI_KIDln Work Units? The slower tasks could just be compiled for 2.2, as they are faster that way. This would immediately speed up the research on Fermi cards (almost double it).
Thanks,

We can reproduce the problem so we can probably fix it.
http://www.gpugrid.net/forum_thread.php?id=2209&nowrap=true#17865

gdf

We can reproduce the problem so we can probably fix it.
http://www.gpugrid.net/forum_thread.php?id=2209&nowrap=true#17865

gdf

It's a problem of a routine in cuda3.1. We have patched it for now.
New application release coming up.

gdf

We have found a very bad bug in CUDA3.1. I have removed the CUDA3.1 application until we can see the extent of the problem.

gdf

On a machine with a mixed Fermi/g200 installation of cards, results on g200 for the FFT are simply wrong. This is a really bad bug.

The only solution is to deliver the CUDA3.1 application to Fermi cards only.

gdf

New application is out for test in beta. Only fermi cards can get the cuda31 app due to the cuda3.1 bug.
This version maintains the level of performance of cuda3.1 for all workunits.

gdf

Picked up two, both failed in about 1 sec!

2599077 1651913 15 Jul 2010 14:35:06 UTC 15 Jul 2010 14:36:31 UTC Error while computing 1.45 0.00 0.00 --- ACEMD beta version v6.30 (cuda31)
2598716 1651698 15 Jul 2010 14:36:31 UTC 15 Jul 2010 14:40:05 UTC Error while computing 0.00 0.00 0.00 --- ACEMD beta version v6.30 (cuda31)

Name 100-GIANNI_TESTDHFR7-1-5-RND4717_0
Workunit 1651913
Created 1 Jul 2010 8:39:18 UTC
Sent 15 Jul 2010 14:35:06 UTC
Received 15 Jul 2010 14:36:31 UTC
Server state Over
Outcome Client error
Client state Compute error
Exit status -40 (0xffffffffffffffd8)
Computer ID 71363
Report deadline 20 Jul 2010 14:35:06 UTC
Run time 1.453125
CPU time 0
stderr out

<core_client_version>6.10.56</core_client_version>
<![CDATA[
<message>
- exit code -40 (0xffffffd8)
</message>
<stderr_txt>
# Using device 0
# There is 1 device supporting CUDA
# Device 0: "GeForce GTX 470"
# Clock rate: 1.41 GHz
# Total amount of global memory: 1341718528 bytes
# Number of multiprocessors: 14
# Number of cores: 112
SWAN: Using synchronization method 0
SWAN : Module load result [.nonbonded.cu.] [200]
SWAN: FATAL : Module load failed


</stderr_txt>
]]>

Validate state Invalid
Claimed credit 0
Granted credit 0
application version ACEMD beta version v6.30 (cuda31)

NVIDIA GeForce GTX 470 (1279MB) driver: 25721
Operating System Microsoft Windows XP

This one also failed,
2598612 1651618 15 Jul 2010 14:40:05 UTC 15 Jul 2010 16:10:07 UTC Error while computing 0.00 0.00 0.00 --- ACEMD beta version v6.30 (cuda31)

They are not even starting,
SWAN: FATAL : Module load failed

Removed, tomorrow another one.
gdf

Trying now with acemdbeta 6.32 for windows.

gdf

Ran one Beta task. It worked on my GTX470 on XP x86, Boinc ver. 6.10.56
Cant find it online yet!
Here are the Boinc details:

16/07/2010 15:00:29 GPUGRID Starting 63-GIANNI_TESTDHFR7-1-5-RND3683_0
16/07/2010 15:00:30 GPUGRID Starting task 63-GIANNI_TESTDHFR7-1-5-RND3683_0 using acemdbeta version 632
16/07/2010 15:05:22 GPUGRID Computation for task 63-GIANNI_TESTDHFR7-1-5-RND3683_0 finished
16/07/2010 15:05:23 GPUGRID Started upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_0
16/07/2010 15:05:23 GPUGRID Started upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_1
16/07/2010 15:05:31 GPUGRID Finished upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_0
16/07/2010 15:05:31 GPUGRID Started upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_2
16/07/2010 15:05:36 GPUGRID Finished upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_1
16/07/2010 15:05:36 GPUGRID Started upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_3
16/07/2010 15:05:47 GPUGRID Finished upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_2
16/07/2010 15:05:47 GPUGRID Started upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_7
16/07/2010 15:05:51 GPUGRID Finished upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_7
16/07/2010 15:05:52 GPUGRID Finished upload of 63-GIANNI_TESTDHFR7-1-5-RND3683_0_3
16/07/2010 15:05:52 GPUGRID Sending scheduler request: To report completed tasks.
16/07/2010 15:05:52 GPUGRID Reporting 1 completed tasks, not requesting new tasks
16/07/2010 15:05:53 GPUGRID Scheduler request completed


GDF, see here too - http://www.gpugrid.net/forum_thread.php?id=2227&nowrap=true#17998

The beta's are working well for GF100 cards, just not for GF104 cards (GTX460). The scientists are now aware of the problem because Beyond reported a difference in Compute Capability between the cards; CC2.0 for GF100 cards, CC2.1 for GF104 cards. GDF said they will try to create a new Beta for the GTX460 on Monday. So check back Monday evening. Apparently the GTX460 works on Collatz. Those wanting to test their card & tune the fans for a nice low temps, you could figure it out there.

GTX470 on XP running a Beta:
2678845 1704781 17 Jul 2010 8:32:57 UTC 17 Jul 2010 11:05:25 UTC
Completed and validated 280.06 276.80 187.28 280.92 ACEMD beta version v6.32 (cuda31)

GTX470 on XP running a Beta:
2678845 1704781 17 Jul 2010 8:32:57 UTC 17 Jul 2010 11:05:25 UTC
Completed and validated 280.06 276.80 187.28 280.92 ACEMD beta version v6.32 (cuda31)

I ran a Folding@home task today while running this Beta. It ran very slow compaired to the other Betas, but at least it ran. You cant do this with non-Fermi cards; the GPUGrid tasks would just fail!

2686592 1708583 18 Jul 2010 Completed and validated 277.58 275.69 187.28 280.92 ACEMD beta version v6.32 (cuda31)
2686539 1707219 18 Jul 2010 Completed and validated 1,442.30 1,438.45 187.28 280.92 ACEMD beta version v6.32 (cuda31)
2686534 1707190 18 Jul 2010 Completed and validated 277.84 276.08 187.28 280.92 ACEMD beta version v6.32 (cuda31)

New beta 6.34 is now out. Hopefully it should work with GTX460 cards, but we could not test it.

gdf

Any more cases of this problem?

gdf

Removed.
Another coming out now.
gdf

application 6.36 is out

gdf

Well,
ok it runs, we can start tests.
it is very slow at least on Windows 7.
Anyone with XP and SWAN_SYNC=0?

gdf

JG4KEZ(Koichi Soraku) has...

half the speed of a GTX480.

gdf

6.36 worked in my GTX460 normally.
GPU load is around 95%.

http://www.gpugrid.net/result.php?resultid=2691958
http://www.gpugrid.net/result.php?resultid=2691640

<core_client_version>6.11.1</core_client_version>
<![CDATA[
<stderr_txt>
# Using device 1
# There are 2 devices supporting CUDA
# Device 0: "GeForce GTX 460"
# Clock rate: 1.40 GHz
# Total amount of global memory: 1073283072 bytes
# Number of multiprocessors: 7
# Number of cores: 56
# Device 1: "GeForce GTX 460"
# Clock rate: 1.40 GHz
# Total amount of global memory: 1073414144 bytes
# Number of multiprocessors: 7
# Number of cores: 56
SWAN: Using synchronization method 0
MDIO ERROR: cannot open file "restart.coor"
# Time per step (avg over 50000 steps): 11.216 ms
# Approximate elapsed time for entire WU: 560.783 s
called boinc_finish

</stderr_txt>
]]>

Perhaps the GTX460's are not using all of their shaders?
Anyway, half the speed of a GTX480 is not bad; a GTX460 costs less than half the price of a GTX480!

It would seem that under Win7 the 1GB version is 10% faster than the 786MB version:

http://www.gpugrid.net/result.php?resultid=2691759
http://www.gpugrid.net/result.php?resultid=2691597

Maybe we can order one and try to optimize for it, it should have done much better.

gdf

It's odd that your clocks are really low.

gdf

Well,
ok it runs, we can start tests.
it is very slow at least on Windows 7.
Anyone with XP and SWAN_SYNC=0?

gdf

Works here too but it's so slow that the 460 will be staying on Collatz for now. In Collatz the GPU usage is 99% and temp 56C. In GPUGRID the GPU usage is 46% and the temp 44C. Win7-32. Why can't we get an app that runs at 95+ percent when every other GPU project does that easily?

http://www.gpugrid.net/result.php?resultid=2691759
http://www.gpugrid.net/result.php?resultid=2691643
http://www.gpugrid.net/result.php?resultid=2691630

Perhaps not just a reporting artefact, either from the driver or the Boinc Version:

Beyond (W7 x86)
NVIDIA GeForce GTX 460 (738MB) driver: 25856
Boinc 6.10.57
Clock rate: 0.81 GHz

JG4KEZ(Koichi Soraku) (XP x86)
[2] NVIDIA GeForce GTX 460 (1023MB) driver: 25896
Clock rate: 1.40 GHz

Bikermatt (W7 x64)
NVIDIA GeForce GTX 460 (993MB) driver: 25896
Clock rate: 1.40 GHz

W7 still reports less RAM than the card actually has.

As we can see from Bikermatt’s results, the Boinc/Driver upgrade would only get it to report the correct speed, and would not improve performance, however it does suggest something:

The difference in performance between an XP system and a W7 system is approximately 0.81/1.4 (to within a few percent) – an XP system takes about 58% of the time to do a similar task, ie 0.81/1.4 of the time.
This is also true of the older GF100 Fermi cards!

This could be a coincidence, but it makes me suspect the update (whatever it is; driver or Boinc version) that allows GPUGrid to display the correct frequency, is actually just displaying what the clock should be, rather than what they actually are, which is still 0.81 under W7 & Vista. This would also explain why the GF100 cards are also slower to about the same ratio.

It looks like Boinc/GPUGrid/Drivers think it is a GF100 card.
Did you do a restart?

Yes, they work, so that is a big +ve.
The downside is that the GTX460's are a little bit slower than expected, but these are only Betas, and that might change with improved applications. It was a good idea to put them on XP - the first XP x64 I have seen in a while ;)

I guess you are not using the swan_sync=0 variable? Try leaving a core free, to see if it improves performance (I expect it will).

Good Luck,

I will be moving the beta to production today.

gdf

Yes, due to the bug in CUDA3.1.

gdf