Advanced search

Message boards : News : More CPU jobs

Author Message
Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49769 - Posted: 3 Jul 2018 | 11:59:53 UTC

...with a new and improved application (Linux only). The current version should eliminate dependencies on gcc and devel libraries.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49770 - Posted: 3 Jul 2018 | 12:26:20 UTC - in response to Message 49769.

By the way, the new app downloads updated libraries. Feel free to reset the project to free up disk space taken by the old ones.

Erich56
Send message
Joined: 1 Jan 15
Posts: 416
Credit: 2,244,307,702
RAC: 2,124,710
Level
Phe
Scientific publications
watwatwatwat
Message 49771 - Posted: 3 Jul 2018 | 19:15:00 UTC

why are these CPU jobs for Linux only, and not for Windows, too?

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 95
Credit: 41,479,788
RAC: 554,849
Level
Val
Scientific publications
wat
Message 49773 - Posted: 4 Jul 2018 | 4:30:49 UTC - in response to Message 49771.

Because they can make the app work under Linux but are not successful yet in creating a Windows app that works.

Erich56
Send message
Joined: 1 Jan 15
Posts: 416
Credit: 2,244,307,702
RAC: 2,124,710
Level
Phe
Scientific publications
watwatwatwat
Message 49774 - Posted: 4 Jul 2018 | 5:38:24 UTC - in response to Message 49773.

Because they can make the app work under Linux but are not successful yet in creating a Windows app that works.

hm, this makes we wonder why it is so much more difficult to create an app for Windows than for Linux ...

further, an easy way to solve this would be to have the Linux app run in a Virtual Machine (like, for example, LHC is doing it for some of it's sub-projects).

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49776 - Posted: 4 Jul 2018 | 6:25:59 UTC - in response to Message 49774.
Last modified: 4 Jul 2018 | 7:35:51 UTC

Making boinc apps is like building a ship in a bottle, in the sense that your tools are very limited and you don't control the environment. In the case of windows the bottle is dark. ;)

Edit. Jokes apart, we would prefer to have the app without VM, which adds opacity and size. QM uses Python, which we try to ship with WUs in the most self-contained way possible. This means that several components have to fall into place. When they don't, the reason is usually guesswork, and search of workarounds.

captainjack
Send message
Joined: 9 May 13
Posts: 135
Credit: 943,558,285
RAC: 5,981
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49780 - Posted: 4 Jul 2018 | 14:41:16 UTC

Erich56 said:

further, an easy way to solve this would be to have the Linux app run in a Virtual Machine


Erich56, if you want to run the Linux app in a Virtual Machine, you can create your own virtual machine, install Linux and BOINC, then run the QC tasks from there. That is what I have done on my Windows machines and it works fine.

tullio
Send message
Joined: 8 May 18
Posts: 34
Credit: 4,355,942
RAC: 100,573
Level
Ala
Scientific publications
wat
Message 49781 - Posted: 4 Jul 2018 | 14:54:39 UTC
Last modified: 4 Jul 2018 | 14:55:06 UTC

I have been running CERN LHC@home Virtual Machines for more than ten years, and I have been rewarded with a CERN Polo Shirt. But yes, they do present some problems. Now your CPU tasks seem to run fine on my old SUN Workstation with SuSE Leap 42.3 Linux.
Tullio

[VENETO] boboviz
Send message
Joined: 10 Sep 10
Posts: 80
Credit: 201,256
RAC: 1,705
Level

Scientific publications
wat
Message 49783 - Posted: 5 Jul 2018 | 9:50:41 UTC - in response to Message 49780.

That is what I have done on my Windows machines and it works fine.


+1
Virtual box on my Win10.
But i think it's not the best solution for performance....

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49784 - Posted: 5 Jul 2018 | 10:02:41 UTC - in response to Message 49783.

As far as I know virtualization is almost native speed these days, especially for computing.

AuxRx
New member
Send message
Joined: 3 Jul 18
Posts: 5
Credit: 176,515
RAC: 10,642
Level

Scientific publications
wat
Message 49785 - Posted: 5 Jul 2018 | 12:01:51 UTC - in response to Message 49784.

The recent batch of CPU WUs seems to be done. Will there be more soon?

Stefan
Volunteer moderator
Project developer
Project scientist
Send message
Joined: 5 Mar 13
Posts: 284
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 49786 - Posted: 5 Jul 2018 | 12:20:39 UTC - in response to Message 49785.

Yes, I am making some now. I'll try to submit new ones today

Stefan
Volunteer moderator
Project developer
Project scientist
Send message
Joined: 5 Mar 13
Posts: 284
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 49787 - Posted: 5 Jul 2018 | 14:24:27 UTC

Sorry, sorry, sorry I messed up due to a small mistake. Had to nuke the WUs. Redoing them now.

AuxRx
New member
Send message
Joined: 3 Jul 18
Posts: 5
Credit: 176,515
RAC: 10,642
Level

Scientific publications
wat
Message 49788 - Posted: 5 Jul 2018 | 14:53:52 UTC - in response to Message 49787.

No issue at all. I'm glad the team communicates openly.

Thank you for the heads-up.

PappaLitto
Send message
Joined: 21 Mar 16
Posts: 366
Credit: 2,640,470,210
RAC: 5,638,721
Level
Phe
Scientific publications
watwat
Message 49789 - Posted: 5 Jul 2018 | 23:55:14 UTC

All of my Stefan CPU WUs are stuck at 10% and I aborted them after about 4 hours. This is the machine (16.04 LTS) that has never had any issues with pretty much any of the WUs.

Here is the Error page: https://www.gpugrid.net/results.php?hostid=424454&offset=0&show_names=0&state=5&appid=

mmonnin
Send message
Joined: 2 Jul 16
Posts: 156
Credit: 259,147,539
RAC: 732,344
Level
Asn
Scientific publications
wat
Message 49790 - Posted: 6 Jul 2018 | 2:35:38 UTC

Holy cow the website is SOO SLOW. I had to use a proxy in Sweden to just get anything to load. I can't even get tasks even though the site says there are plenty.

1 error recently:
http://gpugrid.net/result.php?resultid=18003621

Why is the project requesting 28 GB of disk space? I see a three files stuck downloading that are less than 10kb combined. Project folder and slot folders are nowhere close to 28GB.
GPUGRID 7/5/2018 10:18:01 PM Message from server: Quantum Chemistry, beta test needs 17080.43MB more disk space. You currently have 11529.80 MB available and it needs 28610.23 MB.

These are taking over 4.2-4.5 GB of memory? I thought this was much lower before? That PC has 128GB but Mint split the disk space so not a lot of disk space. But 28 GB is a bit much.

tullio
Send message
Joined: 8 May 18
Posts: 34
Credit: 4,355,942
RAC: 100,573
Level
Ala
Scientific publications
wat
Message 49791 - Posted: 6 Jul 2018 | 7:39:23 UTC

GPUGRID is taking 3.34 GB of disk space on my main Linux host, 3.90 on a Linux laptop. On the same laptop LHC@home is taking 5.75 GB.
Tullio

Stefan
Volunteer moderator
Project developer
Project scientist
Send message
Joined: 5 Mar 13
Posts: 284
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 49792 - Posted: 6 Jul 2018 | 8:09:46 UTC - in response to Message 49791.
Last modified: 6 Jul 2018 | 8:11:48 UTC

Yes this is the stuff I resent to the beta queue I guess. They are much larger molecules so they were crashing on the QM queue cause they ran out of scratch space. I have seen them use up to 18GB scratch space so at the moment I don't know yet how to run these on GPUGRID as it seems to be an issue with many users.

I doubt they were really stuck, they are just much slower to compute.

kain
Send message
Joined: 3 Sep 14
Posts: 131
Credit: 205,729,812
RAC: 201,498
Level
Leu
Scientific publications
watwatwatwatwat
Message 49793 - Posted: 6 Jul 2018 | 10:17:16 UTC

http://gpugrid.net/results.php?hostid=470907

I have a lot of errors :(

tullio
Send message
Joined: 8 May 18
Posts: 34
Credit: 4,355,942
RAC: 100,573
Level
Ala
Scientific publications
wat
Message 49794 - Posted: 6 Jul 2018 | 10:46:21 UTC

Must we update conda?
Please update conda by running

$ conda update -n base conda
Tullio

Jim1348
Send message
Joined: 28 Jul 12
Posts: 555
Credit: 1,182,549,263
RAC: 520,834
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49795 - Posted: 6 Jul 2018 | 10:51:26 UTC - in response to Message 49792.

They are much larger molecules so they were crashing on the QM queue cause they ran out of scratch space. I have seen them use up to 18GB scratch space so at the moment I don't know yet how to run these on GPUGRID as it seems to be an issue with many users.

The most recent Betas have worked OK for me. But I have 32 GB memory, which may help.
http://www.gpugrid.net/results.php?hostid=334241&offset=0&show_names=0&state=0&appid=35

You could set up a special sub-project for the large molecules if you want to.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49796 - Posted: 6 Jul 2018 | 11:38:18 UTC - in response to Message 49795.
Last modified: 6 Jul 2018 | 11:40:14 UTC

We are using QC beta to test large molcules and how much disk space they take. I think they can (temporarily of course) go up to 20 GB of space (!). I am not sure about RAM - they should be < 4 GB.

We'll need to think whether to split the app in large and small.

Regarding conda: it's an "internal" installation. So no, you can't update, and you shouldn't. That message is unfortunate.

Best

kain
Send message
Joined: 3 Sep 14
Posts: 131
Credit: 205,729,812
RAC: 201,498
Level
Leu
Scientific publications
watwatwatwatwat
Message 49797 - Posted: 6 Jul 2018 | 11:47:00 UTC - in response to Message 49796.

We are using QC beta to test large molcules and how much disk space they take. I think they can (temporarily of course) go up to 20 GB of space (!). I am not sure about RAM - they should be < 4 GB.

We'll need to think whether to split the app in large and small.

Regarding conda: it's an "internal" installation. So no, you can't update, and you shouldn't. That message is unfortunate.

Best


Well... So my Threadripper could need up to 160GB of space?! It has just 32GB...

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49798 - Posted: 6 Jul 2018 | 12:15:16 UTC - in response to Message 49797.
Last modified: 6 Jul 2018 | 12:16:05 UTC


Well... So my Threadripper could need up to 160GB of space?! It has just 32GB...


We are talking about DISK space. Only a few WUs will be that big - unless we make a "big" queue.

T
[/quote]

[VENETO] boboviz
Send message
Joined: 10 Sep 10
Posts: 80
Credit: 201,256
RAC: 1,705
Level

Scientific publications
wat
Message 49799 - Posted: 6 Jul 2018 | 12:26:41 UTC - in response to Message 49784.

As far as I know virtualization is almost native speed these days, especially for computing.


Yes, if you are using "hard" virtualization like Esx and Hyper-v.
"Soft" virtualization like VirtualBox or VmPlayer may suffer bottlenecks

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 255
Credit: 704,543,492
RAC: 1,285,560
Level
Lys
Scientific publications
watwat
Message 49800 - Posted: 6 Jul 2018 | 12:26:59 UTC - in response to Message 49796.
Last modified: 6 Jul 2018 | 12:27:24 UTC

We are using QC beta to test large molcules and how much disk space they take.


Toni, if I may ask you, what molecule size are we (roughly) talking about? As you know, because of my son I have personal interest in HCF1 research, and I would like to get a feel how far science is still away from handling that large molecules. Thanks in advance and my apologies for coming up with my personal issues once in a while.
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

tullio
Send message
Joined: 8 May 18
Posts: 34
Credit: 4,355,942
RAC: 100,573
Level
Ala
Scientific publications
wat
Message 49801 - Posted: 6 Jul 2018 | 12:54:37 UTC

I have a degree in Theoretical physics obtained in 1967, but that was related to elementary particle physics. Then in the Nineties, while at Trieste Area Science Park as manager of a UNIX BULL Laboratory I attended a few lectures in the UN Center for Genetic Engineering and Biotechnology on the Density Functional Theory. Since retirement, I have run a few BOINC projects including one on Monte Carlo Method applied to Quantum Chemistry but it no longer exists. This is the first time I am running a project which uses Neural Networks.
Tullio

kain
Send message
Joined: 3 Sep 14
Posts: 131
Credit: 205,729,812
RAC: 201,498
Level
Leu
Scientific publications
watwatwatwatwat
Message 49802 - Posted: 6 Jul 2018 | 13:35:39 UTC - in response to Message 49798.


Well... So my Threadripper could need up to 160GB of space?! It has just 32GB...


We are talking about DISK space. Only a few WUs will be that big - unless we make a "big" queue.

T
[/quote]

I'm also talking about disk space. I'm using 32GB Optane module as boot drive. It's time to change it for something bigger.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49804 - Posted: 6 Jul 2018 | 15:25:04 UTC - in response to Message 49802.

@kain: the disk space is used in the directory BOINC is running. Usually (if you use the distribution installers) it is the in the disk used at the root of the file system, indeed.

STARBASEn
Avatar
Send message
Joined: 17 Feb 09
Posts: 42
Credit: 575,577,900
RAC: 1,277,643
Level
Lys
Scientific publications
watwatwatwatwat
Message 49805 - Posted: 6 Jul 2018 | 16:08:38 UTC

I had to enlarge the root partition to accommodate the QC beta 3.31 since boinc from the Fedora distro by default installs in /var/lib/boinc and runs as a daemon under systemctl. After that, the WU's seemed to run fine but they sure ate up a lot of RAM. Both my 8 core's have 16 GB RAM and I was running them 2 concurrent with 4 cores each. I think only two errored out and the rest completed and validated. Guess I'll have to max my 8 core machines with 32 GB RAM to run the bigger molecules.
____________

Crunching since Feb 2003 (United Devices, Find-a-Drug)

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49806 - Posted: 6 Jul 2018 | 18:35:16 UTC - in response to Message 49805.

Are you able to set the disk limit in the boinc preferences to prevent too many WUs from running?

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 839
Credit: 1,686,015,745
RAC: 1,574,981
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49807 - Posted: 6 Jul 2018 | 18:39:35 UTC - in response to Message 49806.

Easier and more precise simply to set max_concurrent in app_config.xml

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49808 - Posted: 6 Jul 2018 | 18:52:39 UTC - in response to Message 49807.
Last modified: 6 Jul 2018 | 18:56:07 UTC

Great to know, thanks. Actually I was also wondering if boinc respects the disk limits .

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 839
Credit: 1,686,015,745
RAC: 1,574,981
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49809 - Posted: 6 Jul 2018 | 19:08:30 UTC - in response to Message 49808.

Worth it to perform the experiment, certainly. Possibly depends whether it respects the declared space needed (<rsc_disk_bound>), or the actual space used. If the latter, there might be a problem if the actual usage increases gradually during the run - BOINC might only check it when deciding whether to start a(nother) new task. Lots of fun to be had with those possibilities...

STARBASEn
Avatar
Send message
Joined: 17 Feb 09
Posts: 42
Credit: 575,577,900
RAC: 1,277,643
Level
Lys
Scientific publications
watwatwatwatwat
Message 49810 - Posted: 6 Jul 2018 | 19:41:46 UTC

@Toni, Yes boinc does appear to respect the client disk settings as it lets one know if disk space is too low to run certain projects in the event log. I usually set a high arbitrary GB size but the client appears to react to the real amount available in the execution partition and uses the percentage limits to notify user when disk space is too low. I had to readjust the percent limits higher (in the client settings) a few days ago to run the 3.30 app on one of my machines. Probably due to the project directory getting too full. I hate to reset the project and loose WU's but I suppose I will have to eventually.

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 255
Credit: 704,543,492
RAC: 1,285,560
Level
Lys
Scientific publications
watwat
Message 49811 - Posted: 6 Jul 2018 | 19:45:40 UTC - in response to Message 49800.

We are using QC beta to test large molcules and how much disk space they take.


Toni, if I may ask you, what molecule size are we (roughly) talking about? As you know, because of my son I have personal interest in HCF1 research, and I would like to get a feel how far science is still away from handling that large molecules. Thanks in advance and my apologies for coming up with my personal issues once in a while.


I get it. Thank you so much for your help.
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 696
Credit: 4,285,282
RAC: 36
Level
Ala
Scientific publications
watwatwatwat
Message 49812 - Posted: 6 Jul 2018 | 22:09:26 UTC - in response to Message 49800.
Last modified: 6 Jul 2018 | 22:12:00 UTC


Toni, if I may ask you, what molecule size are we (roughly) talking about?


The molecules for QM are max 50 atoms or so. The size is however not very indicative. This is a specific "chemistry-oriented" type of calculations.

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 255
Credit: 704,543,492
RAC: 1,285,560
Level
Lys
Scientific publications
watwat
Message 49813 - Posted: 6 Jul 2018 | 22:49:31 UTC - in response to Message 49812.


Toni, if I may ask you, what molecule size are we (roughly) talking about?


The molecules for QM are max 50 atoms or so. The size is however not very indicative. This is a specific "chemistry-oriented" type of calculations.


Thank you VERY much for that line, I really appreciate that. I was already afraid of being a constant bother. Of course I understand that we are still years or even decades away from handling huge proteins like HCF1 and I don't want to be obtrusive. Having said that, I would like to keep sight of those long term targets.

Thanks again... if I may, I will get back to you with this question in a couple of years. But I am glad that Gpugrid and its team is more than just being "exclusively academic". There actually is a vision of the future we can believe in.
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

tullio
Send message
Joined: 8 May 18
Posts: 34
Credit: 4,355,942
RAC: 100,573
Level
Ala
Scientific publications
wat
Message 49814 - Posted: 6 Jul 2018 | 23:18:18 UTC

On an EDX online course on quantum computers which I followed recently there was a professor at Dartmouth University who uses a quantum computer to do quantum chemistry calculations.
Tullio

Stefan
Volunteer moderator
Project developer
Project scientist
Send message
Joined: 5 Mar 13
Posts: 284
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 49820 - Posted: 9 Jul 2018 | 7:47:14 UTC - in response to Message 49813.

Hey JoergF, while we are not doing proteins with QM yet, (some other groups are trying to do that with networks), what we are calculating is directly related to drug design so I think it is very relevant.
To do whole proteins at QM level accuracy I think will take quite a few more years.

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 255
Credit: 704,543,492
RAC: 1,285,560
Level
Lys
Scientific publications
watwat
Message 49822 - Posted: 9 Jul 2018 | 12:27:44 UTC - in response to Message 49820.
Last modified: 9 Jul 2018 | 12:30:02 UTC

Thank you very much. Which kind of contribution will help you most in order to make progress on proteins (in the long run of course)? Because I am just considering whether to buy an additional GPU or CPU this autumn.
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

Stefan
Volunteer moderator
Project developer
Project scientist
Send message
Joined: 5 Mar 13
Posts: 284
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 49823 - Posted: 9 Jul 2018 | 13:03:52 UTC - in response to Message 49822.

We as a group are not really focusing on applying QM to proteins. The problem being double:

a) QM-trained neural networks are much faster than QM but still slower than MD (molecular dynamics). MD which we can use already is in many cases too slow to study big proteins. So I personally don't see much of a reason yet to simulate proteins with QM.

b) There are other groups working on applying them to proteins but this will be a very difficult challenge over many years.

My focus right now is to instead improve MD so that it's more reliable in its predictions. MD is done in GPU in this project and QM for the moment on CPU. But it would be hard for me to tell you which to prioritize as both really help in the end.

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 255
Credit: 704,543,492
RAC: 1,285,560
Level
Lys
Scientific publications
watwat
Message 49824 - Posted: 9 Jul 2018 | 16:20:54 UTC - in response to Message 49823.
Last modified: 9 Jul 2018 | 16:24:38 UTC

Thank you... no problem. So we just keep on crunching on all sides and see where the road leads us to. :)
By the way, congratulations to all of you guys at Gpugrid on your great work.
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

Profile Chilean
Avatar
Send message
Joined: 8 Oct 12
Posts: 78
Credit: 143,075,554
RAC: 282,554
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwat
Message 49825 - Posted: 10 Jul 2018 | 12:46:41 UTC

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid. I'm thinking of leaving it crunching these instead of Rosetta (the rest of my PCs all run Windows...). Hope it helps!
____________

PappaLitto
Send message
Joined: 21 Mar 16
Posts: 366
Credit: 2,640,470,210
RAC: 5,638,721
Level
Phe
Scientific publications
watwat
Message 49830 - Posted: 10 Jul 2018 | 15:26:11 UTC - in response to Message 49825.
Last modified: 10 Jul 2018 | 15:26:36 UTC

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid. I'm thinking of leaving it crunching these instead of Rosetta (the rest of my PCs all run Windows...). Hope it helps!

Wow! That's a lot of compute!

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 255
Credit: 704,543,492
RAC: 1,285,560
Level
Lys
Scientific publications
watwat
Message 49831 - Posted: 10 Jul 2018 | 15:29:52 UTC - in response to Message 49830.

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid. I'm thinking of leaving it crunching these instead of Rosetta (the rest of my PCs all run Windows...). Hope it helps!

Wow! That's a lot of compute!


Epyc with 48 threads ... I go green with envy :-))
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

Thomas
Send message
Joined: 23 Feb 17
Posts: 9
Credit: 241,284,374
RAC: 966,827
Level
Leu
Scientific publications
wat
Message 49834 - Posted: 10 Jul 2018 | 17:39:17 UTC - in response to Message 49831.

48 QM WUs are 192 (CPU) threads. I need 4 computers to reach that.

Profile Chilean
Avatar
Send message
Joined: 8 Oct 12
Posts: 78
Credit: 143,075,554
RAC: 282,554
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwat
Message 49839 - Posted: 10 Jul 2018 | 21:16:21 UTC - in response to Message 49834.
Last modified: 10 Jul 2018 | 21:17:57 UTC

48 QM WUs are 192 (CPU) threads. I need 4 computers to reach that.


I just realized from your comment that it actually crunches 12 WUs at a time (I just saw all 48 threads running @ 100% immediately thinking it was running 48 WUs just like Rosetta)


I am not a smart man.
____________

morgan
Send message
Joined: 25 Sep 08
Posts: 1
Credit: 16,977,533
RAC: 1,242
Level
Pro
Scientific publications
watwatwatwatwatwatwatwatwat
Message 49840 - Posted: 10 Jul 2018 | 21:23:02 UTC

Well i run QC on a 2core pc just for fun :-)

But now i get; No task sent
This computer has finished a daily quota of 8 task


That´s not fun!!

Profile Retvari Zoltan
Avatar
Send message
Joined: 20 Jan 09
Posts: 1903
Credit: 12,163,917,169
RAC: 2,633,673
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49842 - Posted: 10 Jul 2018 | 21:43:24 UTC - in response to Message 49825.
Last modified: 10 Jul 2018 | 21:44:21 UTC

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid.
To everybody using hyper-threaded CPUs for crunching:
You should test how well the given app scales with HT on or off on your system. The other approach is leave HT on, but lower the percentage of the usable CPUs in BOINC manager (down to 50%). Too many simultaneous memory intensive apps would cause too many cache misses, resulting in degraded combined performance. With HT off (or by setting the usable CPUs to 50%) calculation time should be halved (due that two threads have one FPU). If it's more than a half, then the number of usable CPUs could be increased, while the RAC has risen accordingly (= in a direct ratio).
I can't test it myself until the Windows app has been released, but I'm interested.
A simultaneous GPU task also could degrade the performance of the CPU tasks and vice versa.

mmonnin
Send message
Joined: 2 Jul 16
Posts: 156
Credit: 259,147,539
RAC: 732,344
Level
Asn
Scientific publications
wat
Message 49843 - Posted: 10 Jul 2018 | 23:28:08 UTC

Most tasks benefit from HT but I only recall one doing better overall with HT off on my 2670v1s.

The new Threadrippers with 4 die but only 2 with direct memory access, meaning 2 die have to always make a hop to access memory, would make for some interesting testing.

AuxRx
New member
Send message
Joined: 3 Jul 18
Posts: 5
Credit: 176,515
RAC: 10,642
Level

Scientific publications
wat
Message 49856 - Posted: 13 Jul 2018 | 14:09:11 UTC - in response to Message 49842.
Last modified: 13 Jul 2018 | 14:13:35 UTC

I have a related question I cannot answer myself.

I am seeing very unsteady CPU and memory usage with QC. CPU% drops to 70-80%, with MEM% dropping as well, (however they are defined by ubuntu) after a few seconds, then both peak again.

1) Is this a function of the algorithm or a limitation of my system?

2) Can I limit the number of cores used by QC?

As it happens I've found an issue today where BOINC crunches 3 WCG and one 4 core QC task at the same time on a four core processor. CPU usage has been stuck at 7.x for an hour at least.

QC uses 4 cores by default and I haven't found an option to limited the number of cores. Also, QC seems to be hogging my CPU at most times, eventhough it has been added a week ago. This has led to a bottleneck with WCG tasks due soon.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 555
Credit: 1,182,549,263
RAC: 520,834
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49857 - Posted: 13 Jul 2018 | 17:13:51 UTC - in response to Message 49856.

2) Can I limit the number of cores used by QC?

As it happens I've found an issue today where BOINC crunches 3 WCG and one 4 core QC task at the same time on a four core processor. CPU usage has been stuck at 7.x for an hour at least.

QC uses 4 cores by default and I haven't found an option to limited the number of cores. Also, QC seems to be hogging my CPU at most times, eventhough it has been added a week ago. This has led to a bottleneck with WCG tasks due soon.

Use an "app_config.xml" file to limit the number of cores per work unit, and also the number of QC work units running if you wish.
http://www.gpugrid.net/forum_thread.php?id=4748&nowrap=true#49369

I have found that QC is tough on resources too. Even though I reserved a CPU core to support a GTX 1070 on Folding, running QC still caused a drop in Folding points, showing that the GPU was being starved for CPU support.

To fix that, I now run only six cores of my i7-4770 on CPU work, and leave two cores to support the GPU. But even that was not enough, so I run 4 cores on QC (two work units running two cores each) with the other two on LHC/native ATLAS. That frees up enough CPU resources so that I see only a minimal drop in Folding points.

AuxRx
New member
Send message
Joined: 3 Jul 18
Posts: 5
Credit: 176,515
RAC: 10,642
Level

Scientific publications
wat
Message 49858 - Posted: 13 Jul 2018 | 19:30:15 UTC - in response to Message 49857.

Use an "app_config.xml" file to limit the number of cores per work unit, and also the number of QC work units running if you wish.


Thanks Jim1348, I just tried, but without improvement. Seems to be connected to the algorithm.

mmonnin
Send message
Joined: 2 Jul 16
Posts: 156
Credit: 259,147,539
RAC: 732,344
Level
Asn
Scientific publications
wat
Message 49859 - Posted: 14 Jul 2018 | 3:03:35 UTC - in response to Message 49858.

Use an "app_config.xml" file to limit the number of cores per work unit, and also the number of QC work units running if you wish.


Thanks Jim1348, I just tried, but without improvement. Seems to be connected to the algorithm.


You must tell BOINC to reread configs to pick up the changes. Tasks already downloaded will still say 4c even. Only new ones will say 1c or 2c but all will run at your new setting.

Its a BOINC thing to sometimes squeeze in more tasks than cores. I've seen it happen on my 3570k when a single threaded task completes and a 4c tasks starts it will show more running for a but but it eventually corrects.

AuxRx
New member
Send message
Joined: 3 Jul 18
Posts: 5
Credit: 176,515
RAC: 10,642
Level

Scientific publications
wat
Message 49860 - Posted: 14 Jul 2018 | 14:03:48 UTC - in response to Message 49859.

You must tell BOINC to reread configs to pick up the changes. Tasks already downloaded will still say 4c even. Only new ones will say 1c or 2c but all will run at your new setting.


I did, but it still required a reboot. Tasks that were 4 core previously appeared as x core after a reboot and were crunched as such as well. Credit might take a hit, but I didn't mind for this test.

Its a BOINC thing to sometimes squeeze in more tasks than cores. I've seen it happen on my 3570k when a single threaded task completes and a 4c tasks starts it will show more running for a but but it eventually corrects.


I thought that was the issue, but it wasn't. I even suspended one/several/all QC task, but if/when BOINC could start another task it always did, despite CPU% >400%. CPU% stayed >700% (i.e. 7 tasks running) for an hour plus.

It is working now.

Post to thread

Message boards : News : More CPU jobs