Advanced search

Message boards : Graphics cards (GPUs) : Important question on BOINC and a machine with Fermi/G200 cards

Author Message
Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1957
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 19062 - Posted: 27 Oct 2010 | 9:35:46 UTC

Can you confirm that you if you have multiple GPUs installed BOINC only gives work to the latest compute capability one?

That is, if you have a GTX280 and a GTX480 cards, only the GTX480 will receive work. Is this true?

gdf

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,605,911,851
RAC: 8,679,018
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19064 - Posted: 27 Oct 2010 | 9:59:58 UTC - in response to Message 19062.

Can you confirm that you if you have multiple GPUs installed BOINC only gives work to the latest compute capability one?

That is, if you have a GTX280 and a GTX480 cards, only the GTX480 will receive work. Is this true?

gdf

When the BOINC client requests work from the server, it specifies which of three main hardware classes it needs work for - CPU, NVidia GPU, and/or ATI GPU. It also specifies the details of the devices installed in your computer.

I believe that the BOINC server will allocate work - specifically, by app_version - to what the server believes is the most capable device. So in your example, the GPUGrid server would allocate work using the 6.11 (cuda31) application (assuming Windows), because the GTX480 is the more capable device.

Similarly, the BOINC client will, by default, only use the most capable device for crunching - so that 6.11 (cuda31) work will be processed, as intended, by the GTX480.

But it is possible to set a configuration flag <use_all_gpus>, and many volunteers do this. In this case, work would, additionally, be crunched on the GTX280. But my understanding is that both NVidia GPUs would be allocated work from a single, shared, cache: so the GTX280 (or any lesser card) would also be asked to run the 6.11 (cuda31) application. I don't think there would be any way to specify that two different NVidia GPUs in a single host could receive separate/distinct tasks, or run separate/distinct applications.

Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1957
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 19065 - Posted: 27 Oct 2010 | 10:05:25 UTC - in response to Message 19064.

So with <use_all_gpus> both cards receive work. Is it?

gdf

Profile skgiven
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19067 - Posted: 27 Oct 2010 | 10:15:24 UTC - in response to Message 19065.
Last modified: 27 Oct 2010 | 11:09:31 UTC

Just confirming that this is the case and it has been observed several times here. We presently recommend grouping Fermi cards only with other Fermi's. All non-Fermi cards can all crunch the same 6.05/6.04 work; CC 1.1, CC 1.2 and CC 1.3 cards.

With <use_all_gpus> and a mix of Fermi and non-Fermi, both cards will get Fermi tasks. The non-Fermi card usually Fails the task fairly quickly.

The only way I can think of to use both cards in the same physical system would be to setup a Virtual Machine, and specify which card each machine uses; running one instance of Boinc on the standard system and one on the VM, each with their own configurations. You can specify which card to use; GPU0 or GPU1...

Should anyone have a GT420 for example, in the same system as a GTX295, they would need to just use the GTX295; disabling the use of CUDA for the GT420 using NVidia control panel might do the trick, or setting up the config file to just use GPU1 and not GPU0.

Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1957
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 19068 - Posted: 27 Oct 2010 | 12:55:29 UTC - in response to Message 19067.

Ok,
thanks.
This is clear enough.

gdf

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,605,911,851
RAC: 8,679,018
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19077 - Posted: 28 Oct 2010 | 15:09:50 UTC

Is there any technical reason why the v6.11 Fermi code could not be combined with the v6.04/.05 code into a single 'fat' executable which will run correctly on both types of hardware? This has been achieved by the NVidia programmers who supplied a 'fat' Fermi executable for SETI@home - I have verified that their v6.10 Fermi executable runs correctly on compute capability 1.1 cards.

(Explanation from a third-party programmer:

It seems the Stock build must have contained multiple compiler flags ( '-gencode...' ) for different architectures ....

.... Newest Cuda 3 drivers use JIT PTX compilation on the fly for generating target device specific binaries, where none was embedded for the specific device compute capability. I embedded no explicit v1.1 compute capability kernel binaries or PTX code .... [and the app failed]

If a quick rebuild including special v1.1 flags then works .... [it did] ....)

Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1957
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 19112 - Posted: 30 Oct 2010 | 11:44:55 UTC - in response to Message 19077.
Last modified: 30 Oct 2010 | 11:46:29 UTC

I have changed the server policy.
Now all cards with the appropriate driver should be able to get the CUDA3.1 application. G200 cards installed together with Fermi cards will not work properly until we will release a cuda3.2 app.

gdf

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,605,911,851
RAC: 8,679,018
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19117 - Posted: 30 Oct 2010 | 13:34:32 UTC - in response to Message 19112.

Hang on a second.

What, exactly, have you done here?

Here is the <coprocs> section from a recent work request from my host 71413:

<coprocs>
<coproc_cuda>
<count>1</count>
<name>GeForce 9800 GTX/9800 GTX+</name>
<req_secs>5482.275500</req_secs>
<req_instances>0.000000</req_instances>
<estimated_delay>0.000000</estimated_delay>
<drvVersion>26089</drvVersion>
<cudaVersion>3020</cudaVersion>
<totalGlobalMem>521732096</totalGlobalMem>
<sharedMemPerBlock>16384</sharedMemPerBlock>
<regsPerBlock>8192</regsPerBlock>
<warpSize>32</warpSize>
<memPitch>2147483647</memPitch>
<maxThreadsPerBlock>512</maxThreadsPerBlock>
<maxThreadsDim>512 512 64</maxThreadsDim>
<maxGridSize>65535 65535 1</maxGridSize>
<totalConstMem>65536</totalConstMem>
<major>1</major>
<minor>1</minor>
<clockRate>1890000</clockRate>
<textureAlignment>256</textureAlignment>
<deviceOverlap>1</deviceOverlap>
<multiProcessorCount>16</multiProcessorCount>
</coproc_cuda>
</coprocs>

I've picked out the lines which the scheduler will have used to assign the app_version. With a compute capability of 1.1 (major/minor), this host should still be running the v6.05 application (??? shouldn't it?), even though the driver has already been upgraded for non-GPUGrid reasons - specifically, testing other BOINC applications which are already multi-architecture compiled.

There's no Fermi card in this host to complicate matters, yet the work issued in response to this request was tagged for v6.11, and indeed triggered a first-time download of the v6.11 application:

30/10/2010 12:45:31 GPUGRID Started download of acemd2_6.11_windows_intelx86__cuda31.exe
30/10/2010 12:45:49 GPUGRID Finished download of acemd2_6.11_windows_intelx86__cuda31.exe

(and in the process, validated the 'new_app_version' handling of a specialised BOINC client I'm testing):

30/10/2010 12:45:29 [aDCF] in handle_scheduler_reply(), new app_version aDCF initialised: acemd2 611 (cuda31) windows_intelx86. Defaulting to Project DCF 6.050110

I'll see how this runs when the current task finishes, in about 10 hours: but preliminary indications are that you may just have excluded every single non-fermi card with updated drivers from this project until the 3.2 app is ready for distribution. If that is the case, surely the server configuration change should have been deferred until that application is ready?

Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1957
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 19119 - Posted: 30 Oct 2010 | 14:28:30 UTC - in response to Message 19117.

Cuda3.1 was fine already for G200. The problem was only on mixed configurations fermi/g200, but the server limitation was ineffective exactly in that case if people had used use_all_gpus, so I removed it.
gdf

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,605,911,851
RAC: 8,679,018
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19120 - Posted: 30 Oct 2010 | 14:35:00 UTC - in response to Message 19119.

OK. Thanks for the reassurance. We'll see how it gets on with my G92b overnight, then.

Profile skgiven
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19126 - Posted: 30 Oct 2010 | 18:33:03 UTC - in response to Message 19120.
Last modified: 30 Oct 2010 | 18:51:33 UTC

Why use 260.89 rather than 260.99?

Anyway, this is realy a thread for those with a Fermi and a 200 series card in the same system, which will not work together until the same system until next week some time, hopefully.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,605,911,851
RAC: 8,679,018
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19127 - Posted: 30 Oct 2010 | 18:50:09 UTC - in response to Message 19126.

Why use 260.89 rather than 260.99?

Because I've shut down for a driver upgrade once in this sequence already (when 260.89 first came out), and it's more efficient to leave it running than to shut down again for a second upgrade.

Unless you can point to a visible improvement between the two?

MarkJ
Volunteer moderator
Volunteer tester
Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19129 - Posted: 31 Oct 2010 | 3:23:22 UTC - in response to Message 19127.

Why use 260.89 rather than 260.99?

Because I've shut down for a driver upgrade once in this sequence already (when 260.89 first came out), and it's more efficient to leave it running than to shut down again for a second upgrade.

Unless you can point to a visible improvement between the two?


When I installed .99 over the top of .89 (did a clean install) it didn't require a reboot. But you will need to shut down BOINC while you do it.

As for advantages I haven't seen anything mentioned that suggests it fixes up anything other than some games. The installer change wasn't mentioned either (apart from the stuff about .89 installer change).
____________
BOINC blog

Profile skgiven
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19132 - Posted: 31 Oct 2010 | 7:33:08 UTC - in response to Message 19129.

Did you observe any speed loss (frequency) with your 9800GT's?

Profile Beyond
Avatar
Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19136 - Posted: 31 Oct 2010 | 13:52:00 UTC - in response to Message 19132.
Last modified: 31 Oct 2010 | 13:52:59 UTC

Did you observe any speed loss (frequency) with your 9800GT's?

Speaking of speed, I'm seeing a HUGE slowdown on all 3 of my GT 240 cards with the 6.11 app. Seems to be in the neighborhood of 60% but very few WUs have been completed because they're running so slow. GPU usage is still above 90% so that's not the problem. I hope we can go back to the 6.05 app quickly as 6.11 seems to be a big waste of electricity. They're all on WinXP64.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,605,911,851
RAC: 8,679,018
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19143 - Posted: 31 Oct 2010 | 18:03:32 UTC - in response to Message 19132.

Did you observe any speed loss (frequency) with your 9800GT's?

First task reported with v6.11 on 9800GT: host 43362

I don't think I'd call that runtime variation significant until we get more data. But at least it ran, and validated: let's hope the HIVPR_n1 I'm running on another machine is as successful.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1576
Credit: 5,605,911,851
RAC: 8,679,018
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 19147 - Posted: 31 Oct 2010 | 20:11:43 UTC

Sorry guys, HIVPR_n1 crashed as usual: host 45218

Back to the abort button.

Post to thread

Message boards : Graphics cards (GPUs) : Important question on BOINC and a machine with Fermi/G200 cards

//