Advanced search

Message boards : News : CPU jobs on Linux

Author Message
Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1909
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 48820 - Posted: 5 Feb 2018 | 19:30:28 UTC

Hi, we need more CPUs on Linux to run QM simulations. Anybody can help?

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 82
Credit: 9,586,188
RAC: 2,271
Level
Ser
Scientific publications
wat
Message 48821 - Posted: 5 Feb 2018 | 20:05:56 UTC

Do you want to make them more appealing to crunch?? Take the QMML tasks OFF the Boinc CreditNew credit award mechanism and assign them fixed values like you do for the gpu tasks.

biodoc
Send message
Joined: 26 Aug 08
Posts: 114
Credit: 772,027,133
RAC: 551,520
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 48823 - Posted: 5 Feb 2018 | 20:47:06 UTC

Sure, I can help.

Matt Kowal
Send message
Joined: 27 May 14
Posts: 1
Credit: 1,188,500
RAC: 0
Level
Ala
Scientific publications
wat
Message 48824 - Posted: 5 Feb 2018 | 21:18:15 UTC
Last modified: 5 Feb 2018 | 21:23:18 UTC

This forum news post was syndicated to your Twitter account, however, the link is broken.

Relevant post: https://twitter.com/gpugrid/status/960604705171808256

The link resolves to https://www.gpugrid.net/extra_arg_utm_source.html

I have reposted your call to the BOINC subreddit

klepel
Send message
Joined: 23 Dec 09
Posts: 151
Credit: 1,972,629,934
RAC: 730,937
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 48825 - Posted: 5 Feb 2018 | 21:45:30 UTC - in response to Message 48821.
Last modified: 5 Feb 2018 | 21:47:43 UTC

Do you want to make them more appealing to crunch?? Take the QMML tasks OFF the Boinc CreditNew credit award mechanism and assign them fixed values like you do for the gpu tasks.

+1

PS Unfortunatelly, they crash my computer with my strongest GPU frequently, so I will not run them on this computer.

John C MacAlister
Send message
Joined: 17 Feb 13
Posts: 179
Credit: 132,357,411
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwat
Message 48826 - Posted: 5 Feb 2018 | 22:51:50 UTC - in response to Message 48820.

Unfortunately, I gave up on Linux and run Win 10.


Hi, we need more CPUs on Linux to run QM simulations. Anybody can help?


____________
John

Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1909
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 48827 - Posted: 5 Feb 2018 | 23:23:41 UTC - in response to Message 48821.

Do you want to make them more appealing to crunch?? Take the QMML tasks OFF the Boinc CreditNew credit award mechanism and assign them fixed values like you do for the gpu tasks.



We are testing this.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 82
Credit: 9,586,188
RAC: 2,271
Level
Ser
Scientific publications
wat
Message 48828 - Posted: 6 Feb 2018 | 0:22:33 UTC - in response to Message 48826.

I've had good luck with the Linux apps up until the recent gpu application errors that started this month. The cpu tasks ran fine.

I and others have voiced our displeasure with the credit awarded for the flops used for the QM tasks in this thread.
New Student and QMML Project

Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1909
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 48833 - Posted: 6 Feb 2018 | 9:02:58 UTC - in response to Message 48828.

we are testing different credit systems now.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 638
Credit: 4,273,184
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 48834 - Posted: 6 Feb 2018 | 10:45:39 UTC - in response to Message 48833.
Last modified: 6 Feb 2018 | 10:46:23 UTC

We don't use CreditNew but the previous credit system. In any case, two changes were made yesterday:

* CPU threads are limited to 4 (you should still be able to crunch multiple WUs at once, please check)
* Credits should be in line with other projects'

Let us know.

Profile Bikermatt
Send message
Joined: 8 Apr 10
Posts: 37
Credit: 2,114,242,098
RAC: 2,066,158
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 48836 - Posted: 6 Feb 2018 | 12:54:52 UTC - in response to Message 48834.


* CPU threads are limited to 4 (you should still be able to crunch multiple WUs at once, please check)

Let us know.


Boinc is still assigning all of my 32 threads to one task even though it is only using 4.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 517
Credit: 1,137,278,461
RAC: 20,414
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 48837 - Posted: 6 Feb 2018 | 13:29:06 UTC - in response to Message 48820.

Hi, we need more CPUs on Linux to run QM simulations. Anybody can help?

I have three machines on it, but I can run only one work unit at a time on average. That is because when any more start up at once, they error out, as has been discussed before. And I run two cores per work unit for efficiency. But if you could solve the start-up problem, I could run more.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 517
Credit: 1,137,278,461
RAC: 20,414
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 48838 - Posted: 6 Feb 2018 | 13:55:26 UTC

I don't see that you have made an announcement on the BOINC forum yet. The Projects section would probably be best.
http://boinc.berkeley.edu/dev/forum_forum.php?id=11

biodoc
Send message
Joined: 26 Aug 08
Posts: 114
Credit: 772,027,133
RAC: 551,520
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 48839 - Posted: 6 Feb 2018 | 14:24:00 UTC

I have an Intel 2600K with 8 logical cores. I wanted to reserved 2 cores for feeding 2 GPUs that are running Folding@home. I set the computing preference in boinc to use, at most, 80% of processors (6 cores).

Data on 19 WUs before GPUGrid changes (processor usage varied but 5-6 cores on average I think)

Average run time (sec): 3,129.23
Average CPU time (sec): 16,122.19
Average credit per WU: 228.7
Average WU per day: 27.6
PPD: 6314.8

Data on 15 WUs after GPUGrid changes. Processor usage was 4 cores even though set at 6)

Average run time (sec): 3,566.32
Average CPU time (sec): 14,114.28
Average credit per WU: 819.284
Average WU per day: 24.2
PPD: 19848.5

Summary: Processor usage seems to be maxed out at 4. Run time has increased, CPU time has decreased, WU completed per day has decreased and PPD has increased significantly.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 638
Credit: 4,273,184
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 48840 - Posted: 6 Feb 2018 | 14:57:09 UTC - in response to Message 48836.
Last modified: 6 Feb 2018 | 15:09:56 UTC



Boinc is still assigning all of my 32 threads to one task even though it is only using 4.


Ouch. This should not happen. May be fixed now.

mmonnin
Send message
Joined: 2 Jul 16
Posts: 137
Credit: 177,213,592
RAC: 512,425
Level
Ile
Scientific publications
wat
Message 48842 - Posted: 6 Feb 2018 | 15:50:39 UTC - in response to Message 48837.

Hi, we need more CPUs on Linux to run QM simulations. Anybody can help?

I have three machines on it, but I can run only one work unit at a time on average. That is because when any more start up at once, they error out, as has been discussed before. And I run two cores per work unit for efficiency. But if you could solve the start-up problem, I could run more.


I agree. There were too many issues that had not been resolved. On top of that the credit much worse than even CreditNew. I see the credit has been changed just saying there are reasons for the lack of CPU time.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 82
Credit: 9,586,188
RAC: 2,271
Level
Ser
Scientific publications
wat
Message 48845 - Posted: 6 Feb 2018 | 17:48:28 UTC - in response to Message 48834.

Are you sure that you are using an "older" credit mechanism? From the response in the "New Student and QMML" thread from Richard Haselgrove who corrected me in my assumption you might be using an "older" mechanism.

Be careful of your terms. 'CreditNew' has been the default BOINC mechanism since 2010. I suspect this is what GPUGrid is using for these tasks: the support mechanisms for 'even older credit' have been removed from the codebase.

I wonder where you found the older codebase that has been removed that contained the "older" credit award algorithm. If you do in fact have such, I would like access to it. Or have it reinstituted into the BOINC Github codebase.

It would be helpful in persuading the BOINC maintainers that there is in fact a way to return to the older credit algorithm. One of their stated reasons why they said they would not change from CreditNew is that they said they no longer have the original code and can't replicate it.

That said, it looks like the award for QC cpu tasks is much more appealing now.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 82
Credit: 9,586,188
RAC: 2,271
Level
Ser
Scientific publications
wat
Message 48847 - Posted: 6 Feb 2018 | 19:21:26 UTC - in response to Message 48839.

Hmmm ... I just ran a new QC task with the supposed new credit. Not seeing any difference.

Run time 3,292.22
CPU time 12,925.47
Validate state Valid
Credit 110.63

I used 4 cores to generate 110 credits for 54 minutes of compute time.

I can use one core to generate 108 credits for 60 minutes of compute time for SETI CPU tasks.

No reason to run these tasks still for me.

STARBASEn
Avatar
Send message
Joined: 17 Feb 09
Posts: 34
Credit: 453,042,714
RAC: 1,133,460
Level
Gln
Scientific publications
watwatwatwatwat
Message 48850 - Posted: 6 Feb 2018 | 23:50:37 UTC

Since just after mid day yesterday (UTC) I have noticed an increase in credit awarded for the QC WU's. Doing some quick calcs, it appears the increase is about 4.5x of what we were getting. It also appears they are more fixed in value proportional to the size of the WU. My faster machines are getting over 500 credits/hr (4 cores) compute time whereas the slower machines are getting proportionately less/hr and still getting equivalent credit but over a longer period of time than the faster ones.

Well, at least my avg credit per day will not be taking as much of a hit per day as it has been without Linux GPU WU's :).

So far, I have 5 machines with 4 cores each running the QC project and will add one more when I install the memory upgrade on one of the headless systems.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 82
Credit: 9,586,188
RAC: 2,271
Level
Ser
Scientific publications
wat
Message 48851 - Posted: 7 Feb 2018 | 0:28:50 UTC - in response to Message 48850.

I'm not sure the higher credit is not due to the larger molecule size in the latest tasks that Dominik explained to me here.

Trotador
Send message
Joined: 25 Mar 12
Posts: 87
Credit: 1,148,558,105
RAC: 639,987
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwat
Message 48857 - Posted: 7 Feb 2018 | 6:26:25 UTC

No higher credits for the tasks I've crunched yesterday and today.

I stop and will try to understand what is happening.

NUCCpod_NAPTIMELABS_01
Send message
Joined: 18 Aug 17
Posts: 6
Credit: 18,577,273
RAC: 406,792
Level
Pro
Scientific publications
wat
Message 48858 - Posted: 7 Feb 2018 | 6:33:41 UTC - in response to Message 48837.

I've also been having trouble with work units erroring out in this way. I have over 150 cpu cores spread out across a variety of machines, all under linux. Just a few moments ago I attempted to attach to GPUGRID only to have computational error after computational error. I hope this is fixed shortly as I would love to have GPUGRID as one of my default projects.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 82
Credit: 9,586,188
RAC: 2,271
Level
Ser
Scientific publications
wat
Message 48859 - Posted: 7 Feb 2018 | 7:46:25 UTC

Yes, credit is doing something very strange. I got 111 credits for 3292 seconds of cpu time.

klepel got 1362 credits for the same time.

Run time 3,262.28
CPU time 12,834.34
Validate state Valid
Credit 1,361.79

Task 13118994

kain
Send message
Joined: 3 Sep 14
Posts: 120
Credit: 176,509,215
RAC: 306,573
Level
Ile
Scientific publications
watwatwatwatwat
Message 48862 - Posted: 7 Feb 2018 | 12:23:54 UTC - in response to Message 48847.


I used 4 cores to generate 110 credits for 54 minutes of compute time.

I can use one core to generate 108 credits for 60 minutes of compute time for SETI CPU tasks.

No reason to run these tasks still for me.


Are you seriously comparing SETI to GPUGRID?!

mmonnin
Send message
Joined: 2 Jul 16
Posts: 137
Credit: 177,213,592
RAC: 512,425
Level
Ile
Scientific publications
wat
Message 48863 - Posted: 7 Feb 2018 | 12:53:02 UTC - in response to Message 48862.


I used 4 cores to generate 110 credits for 54 minutes of compute time.

I can use one core to generate 108 credits for 60 minutes of compute time for SETI CPU tasks.

No reason to run these tasks still for me.


Are you seriously comparing SETI to GPUGRID?!


CreditNew is CreditNew.

PappaLitto
Send message
Joined: 21 Mar 16
Posts: 321
Credit: 2,052,122,900
RAC: 6,368,789
Level
Phe
Scientific publications
watwat
Message 48864 - Posted: 7 Feb 2018 | 13:33:40 UTC - in response to Message 48863.


I used 4 cores to generate 110 credits for 54 minutes of compute time.

I can use one core to generate 108 credits for 60 minutes of compute time for SETI CPU tasks.

No reason to run these tasks still for me.


Are you seriously comparing SETI to GPUGRID?!


CreditNew is CreditNew.

You cannot compare one project's credit to another.

computezrmle
Send message
Joined: 10 Jun 13
Posts: 5
Credit: 20,203,541
RAC: 201,200
Level
Pro
Scientific publications
wat
Message 48866 - Posted: 7 Feb 2018 | 16:54:10 UTC - in response to Message 48864.

You cannot compare one project's credit to another.

At least it should be comparable as described in the BOINC documentation.
See: http://boinc.berkeley.edu/trac/wiki/CreditNew#Cross-projectversionnormalization

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 82
Credit: 9,586,188
RAC: 2,271
Level
Ser
Scientific publications
wat
Message 48867 - Posted: 7 Feb 2018 | 17:01:28 UTC - in response to Message 48864.


I used 4 cores to generate 110 credits for 54 minutes of compute time.

I can use one core to generate 108 credits for 60 minutes of compute time for SETI CPU tasks.

No reason to run these tasks still for me.


Are you seriously comparing SETI to GPUGRID?!


CreditNew is CreditNew.

You cannot compare one project's credit to another.

One of the stated objectives of CreditNew it to make credit the same across all projects for the same amount of cobblestones used to compute.

mmonnin
Send message
Joined: 2 Jul 16
Posts: 137
Credit: 177,213,592
RAC: 512,425
Level
Ile
Scientific publications
wat
Message 48872 - Posted: 7 Feb 2018 | 18:35:25 UTC

Thus my comment about CreditNew. CPU projects that have higher or lower then typical RAC are most likely using something else besides CreditNew like a fixed credit or another algorithm.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 82
Credit: 9,586,188
RAC: 2,271
Level
Ser
Scientific publications
wat
Message 48873 - Posted: 7 Feb 2018 | 18:46:31 UTC - in response to Message 48872.

Yes I understood your post and sentiment. My post was directed at the other poster's incredulous comment.

This project itself utilize both mechanisms. CreditNew for cpu tasks and fixed credit awards for gpu tasks.

As far as I have been able to find, that is unique among projects. Usually it is either/or not both.

mmonnin
Send message
Joined: 2 Jul 16
Posts: 137
Credit: 177,213,592
RAC: 512,425
Level
Ile
Scientific publications
wat
Message 48876 - Posted: 7 Feb 2018 | 22:11:38 UTC - in response to Message 48873.

Yes I understood your post and sentiment. My post was directed at the other poster's incredulous comment.

This project itself utilize both mechanisms. CreditNew for cpu tasks and fixed credit awards for gpu tasks.

As far as I have been able to find, that is unique among projects. Usually it is either/or not both.


I wasn't referencing you as I didn't quote you. ;)

I agree.

Profile Michael H.W. Weber
Send message
Joined: 9 Feb 16
Posts: 32
Credit: 294,437,438
RAC: 124,215
Level
Asn
Scientific publications
watwat
Message 48893 - Posted: 10 Feb 2018 | 15:13:42 UTC
Last modified: 10 Feb 2018 | 15:18:45 UTC

Would you please list the QC project progress on the server status page as well:

http://www.gpugrid.net/server_status.php

Thanks.

Another issue is that your app does not dynamically allocate CPU cores according to the BOINC settings. Instead it claims all physically present cores. That is a major problem when trying to run computations on the GPU as well because to do so, the GPU project automatically (or I manually) reserve(s) one CPU core per GPU task.
Example: When the BOINC manager is set to use 7 of the 8 cores to do CPU computations, your CPU client grabs all 8 cores (or since recently 2x 4 cores).
That is not acceptable.
Please fix this to attract more people to donate CPU cycles to your project.

Michael.
____________
President of Rechenkraft.net - Germany's first and largest distributed computing organization.

Profile SMTB1963
Avatar
Send message
Joined: 27 Jun 10
Posts: 37
Credit: 215,284,871
RAC: 1,414
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwat
Message 48894 - Posted: 10 Feb 2018 | 17:48:45 UTC - in response to Message 48893.

Another issue is that your app does not dynamically allocate CPU cores according to the BOINC settings. Instead it claims all physically present cores.


I'm seeing this behavior as well. On my Ryzen 1700X system with 2 GPUs, these WUs basically take over all CPUs and throw the GPUs into "Waiting to run".

I suppose one could set max_concurrent in an app_config.xml to fix this...what would be the proper app name to use?

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 82
Credit: 9,586,188
RAC: 2,271
Level
Ser
Scientific publications
wat
Message 48895 - Posted: 10 Feb 2018 | 17:54:26 UTC - in response to Message 48894.

QC is the proper app name. This is how I limit QC to 2 threads per task.

<app>
<name>QC</name>
<max_concurrent>1</max_concurrent>
</app>
<app_version>
<app_name>QC</app_name>
<plan_class>mt</plan_class>
<avg_ncpus>2.000000</avg_ncpus>
<cmdline>--nthreads 2</cmdline>


I also just run 1 task at a time to avoid the starting two tasks at the same time flaw in the application.

[VENETO] boboviz
Send message
Joined: 10 Sep 10
Posts: 53
Credit: 135,377
RAC: 340
Level

Scientific publications
wat
Message 48988 - Posted: 19 Feb 2018 | 8:24:05 UTC

An error on a wu (task 17040682):

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<message>
process exited with code 195 (0xc3, -61)</message>
<stderr_txt>
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
ConnectionError(MaxRetryError("HTTPSConnectionPool(host='repo.continuum.io', port=443): Max retries exceeded with url: /pkgs/main/linux-64/repodata.json.bz2 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fc2a3f56860>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))",),)


Traceback (most recent call last):
File "pre_script.py", line 13, in <module>
raise Exception("Error installing h5py")
Exception: Error installing h5py
08:49:09 (1252): $PROJECT_DIR/miniconda/bin/python exited; CPU time 0.469942
08:49:09 (1252): app exit status: 0x1
08:49:09 (1252): called boinc_finish(195)

Profile Logan Carr
Send message
Joined: 12 Aug 15
Posts: 224
Credit: 41,557,936
RAC: 38,303
Level
Val
Scientific publications
wat
Message 49092 - Posted: 24 Feb 2018 | 0:21:06 UTC - in response to Message 48820.

Hi, we need more CPUs on Linux to run QM simulations. Anybody can help?


okay, I'm going to install Linux right now on my computer and it should be ready tonight or early tomorrow.
____________
Cruncher/Learner in progress.

biodoc
Send message
Joined: 26 Aug 08
Posts: 114
Credit: 772,027,133
RAC: 551,520
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49094 - Posted: 24 Feb 2018 | 12:04:32 UTC - in response to Message 48988.


Traceback (most recent call last):
File "pre_script.py", line 13, in <module>
raise Exception("Error installing h5py")
Exception: Error installing h5py
08:49:09 (1252): $PROJECT_DIR/miniconda/bin/python exited; CPU time 0.469942
08:49:09 (1252): app exit status: 0x1
08:49:09 (1252): called boinc_finish(195)


If you are running the latest distros from Ubuntu or Mint you may need to install the python-support package.

wget http://launchpadlibrarian.net/109052632/python-support_1.0.15_all.deb

sudo dpkg -i python-support_1.0.15_all.deb

kain
Send message
Joined: 3 Sep 14
Posts: 120
Credit: 176,509,215
RAC: 306,573
Level
Ile
Scientific publications
watwatwatwatwat
Message 49096 - Posted: 24 Feb 2018 | 15:57:29 UTC

One of my CPUs is crunching linux jobs, I will add two more next week.

kain
Send message
Joined: 3 Sep 14
Posts: 120
Credit: 176,509,215
RAC: 306,573
Level
Ile
Scientific publications
watwatwatwatwat
Message 49164 - Posted: 16 Mar 2018 | 11:53:15 UTC

I have added three. Athlon 5350, I3 3240, I3 6100. Not the best ones but doing their job. I can add more but I would like to know how many WUs can we expect for the Linux QM app?

BTW ATM just 92 users are crunching QM, that's sad :(

Jim1348
Send message
Joined: 28 Jul 12
Posts: 517
Credit: 1,137,278,461
RAC: 20,414
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49165 - Posted: 16 Mar 2018 | 13:05:14 UTC - in response to Message 49164.
Last modified: 16 Mar 2018 | 13:09:04 UTC

I have added another machine, and now have four on it (2 i7-3770, 1 i7-4790 and 1 Ryzen 1700), with two to four cores per machine allocated via the resource share. The main problem in running them is that when two or more start up at the same time, they error out. That happens mainly during reboots, but otherwise they never start up at the same time. I leave my machines running 24/7, so I don't reboot very often.

And to minimize the problem, you can run with the default 4 cores per work unit and only 4 cores (or less) per machine on average, so that they usually don't start more than one work unit at a time anyway. In that way, it is a manageable problem for me, though it would be best if they fix it. I am sure more people would then be willing to run it.

Also a Windows version would help of course, and for that they had better be seriously thinking about VirtualBox.

captainjack
Send message
Joined: 9 May 13
Posts: 125
Credit: 927,430,520
RAC: 612,039
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49167 - Posted: 16 Mar 2018 | 19:28:12 UTC

Jim 1348 said:

Also a Windows version would help of course, and for that they had better be seriously thinking about VirtualBox.


If you want to go the virtualbox route, you can create your own virtualbox instance, install your favorite flavor of Linux (I chose Ubuntu), make sure that gcc is installed, install BOINC and start running the Linux version of the Quantum Chemistry tasks.

It took me a few tries to figure out that gcc needed to be installed, but now they seem to be running fine.

mmonnin
Send message
Joined: 2 Jul 16
Posts: 137
Credit: 177,213,592
RAC: 512,425
Level
Ile
Scientific publications
wat
Message 49168 - Posted: 17 Mar 2018 | 15:09:22 UTC - in response to Message 49164.

I have added three. Athlon 5350, I3 3240, I3 6100. Not the best ones but doing their job. I can add more but I would like to know how many WUs can we expect for the Linux QM app?

BTW ATM just 92 users are crunching QM, that's sad :(


People aren't running it since it has so many issues that haven't been addressed.

kain
Send message
Joined: 3 Sep 14
Posts: 120
Credit: 176,509,215
RAC: 306,573
Level
Ile
Scientific publications
watwatwatwatwat
Message 49169 - Posted: 17 Mar 2018 | 16:42:59 UTC

Lucky me (not even a single problem on my end)

flashawk
Send message
Joined: 18 Jun 12
Posts: 268
Credit: 1,938,174,347
RAC: 2,976,526
Level
His
Scientific publications
watwatwatwatwatwatwatwat
Message 49171 - Posted: 18 Mar 2018 | 4:07:07 UTC

The main reason why I came here was to put my GPU's to work, my CPU cores are all very busy with CPDN.

mmonnin
Send message
Joined: 2 Jul 16
Posts: 137
Credit: 177,213,592
RAC: 512,425
Level
Ile
Scientific publications
wat
Message 49172 - Posted: 18 Mar 2018 | 12:35:01 UTC - in response to Message 49169.

Lucky me (not even a single problem on my end)


Start two at once..

kain
Send message
Joined: 3 Sep 14
Posts: 120
Credit: 176,509,215
RAC: 306,573
Level
Ile
Scientific publications
watwatwatwatwat
Message 49173 - Posted: 18 Mar 2018 | 12:39:17 UTC - in response to Message 49172.

I understand that. I'm using only 4C or 2c/4t CPUs so I can't start two at once. Probably that's why i don't have any issues :)

Jim1348
Send message
Joined: 28 Jul 12
Posts: 517
Credit: 1,137,278,461
RAC: 20,414
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49174 - Posted: 18 Mar 2018 | 13:37:13 UTC - in response to Message 49167.

Jim 1348 said:
Also a Windows version would help of course, and for that they had better be seriously thinking about VirtualBox.


If you want to go the virtualbox route, you can create your own virtualbox instance, install your favorite flavor of Linux (I chose Ubuntu), make sure that gcc is installed, install BOINC and start running the Linux version of the Quantum Chemistry tasks.

Thanks, but I have five Ubuntu machines. I was offering that only as advice if they want to increase their processing power. They need to enlist the Windows users.

Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1909
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 49179 - Posted: 20 Mar 2018 | 21:53:21 UTC - in response to Message 49174.

The linux QM app runs fine but it requires some packages to be installed. People should expect infinite number of workunits...

We will run this for ever.

kain
Send message
Joined: 3 Sep 14
Posts: 120
Credit: 176,509,215
RAC: 306,573
Level
Ile
Scientific publications
watwatwatwatwat
Message 49180 - Posted: 20 Mar 2018 | 22:03:00 UTC - in response to Message 49179.

The linux QM app runs fine but it requires some packages to be installed. People should expect infinite number of workunits...

We will run this for ever.



Wow, wow, wow! :D
I will add some more CPUs.

When can we expect some info about scientific results created with this app?

Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1909
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 49193 - Posted: 22 Mar 2018 | 10:59:44 UTC - in response to Message 49180.


When can we expect some info about scientific results created with this app?


Months away, we have just started.

(Ryle)
Send message
Joined: 7 Jun 09
Posts: 18
Credit: 781,382,646
RAC: 52,685
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 49197 - Posted: 22 Mar 2018 | 13:16:35 UTC

Could someone who knows how, make a description of exactly which dependencies are needed to make the QC app run flawlessly, in the FAQ section? I think it is a good place for it to be described.

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 239
Credit: 550,771,414
RAC: 1,830,271
Level
Lys
Scientific publications
watwat
Message 49204 - Posted: 24 Mar 2018 | 9:23:17 UTC - in response to Message 49173.
Last modified: 24 Mar 2018 | 9:31:11 UTC

I understand that. I'm using only 4C or 2c/4t CPUs so I can't start two at once. Probably that's why i don't have any issues :)


Once those issues have been solved I will gladly set a 16/32t Threadripper on the QC tasks to shovel them away ;)
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 239
Credit: 550,771,414
RAC: 1,830,271
Level
Lys
Scientific publications
watwat
Message 49205 - Posted: 25 Mar 2018 | 8:57:18 UTC
Last modified: 25 Mar 2018 | 9:04:26 UTC

And to minimize the problem, you can run with the default 4 cores per work unit and only 4 cores (or less) per machine on average, so that they usually don't start more than one work unit at a time anyway. In that way, it is a manageable problem for me, though it would be best if they fix it. I am sure more people would then be willing to run it.


I have set up a i3 Sandy Bridge machine (2c/4t) on Ubuntu and use the default config as suggested above, but all tasks report a calculation error after just 1-3 minutes. Any idea what goes wrong here?

http://www.gpugrid.net/result.php?resultid=17336453
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

captainjack
Send message
Joined: 9 May 13
Posts: 125
Credit: 927,430,520
RAC: 612,039
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49206 - Posted: 25 Mar 2018 | 12:18:12 UTC

JoergF

Try installing gcc and see if that helps.

In a terminal session

sudo apt-get install gcc

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 239
Credit: 550,771,414
RAC: 1,830,271
Level
Lys
Scientific publications
watwat
Message 49207 - Posted: 25 Mar 2018 | 18:36:21 UTC - in response to Message 49206.

JoergF

Try installing gcc and see if that helps.

In a terminal session
sudo apt-get install gcc


Thank you... I have installed it. However I am not able to test it as there seems to be a daily quota of 2 tasks per computer and I have to wait for the next day. Really, in view of of >10.000 unsent QC tasks, that limitation is somewhat surprising.
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

captainjack
Send message
Joined: 9 May 13
Posts: 125
Credit: 927,430,520
RAC: 612,039
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49208 - Posted: 25 Mar 2018 | 20:34:23 UTC

JoergF said:

However I am not able to test it as there seems to be a daily quota of 2 tasks per computer and I have to wait for the next day


There is a daily limit if a cruncher starts turning more than the usual number of errors. Once you start turning in valid tasks, the daily limit will go back up. I have been caught in it a few times when running test tasks.

Hopefully you will get better results tomorrow.

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 239
Credit: 550,771,414
RAC: 1,830,271
Level
Lys
Scientific publications
watwat
Message 49209 - Posted: 26 Mar 2018 | 6:53:19 UTC - in response to Message 49208.

Thank you.. seems to work now :)
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

[VENETO] boboviz
Send message
Joined: 10 Sep 10
Posts: 53
Credit: 135,377
RAC: 340
Level

Scientific publications
wat
Message 49211 - Posted: 26 Mar 2018 | 19:13:52 UTC - in response to Message 49179.

People should expect infinite number of workunits...
We will run this for ever.


Oh, well.
So we dream for a opencl client (or an sse/avx cpu optimization)

Jim1348
Send message
Joined: 28 Jul 12
Posts: 517
Credit: 1,137,278,461
RAC: 20,414
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49212 - Posted: 26 Mar 2018 | 20:07:51 UTC - in response to Message 49211.

Oh, well.
So we dream for a opencl client (or an sse/avx cpu optimization)

As you probably know, the FMA version of TN-Grid is faster than the AVX version. At least that was my result comparing a Ryzen 1700 (FMA) to an i7-4770 (AVX), both on Ubuntu.

[VENETO] boboviz
Send message
Joined: 10 Sep 10
Posts: 53
Credit: 135,377
RAC: 340
Level

Scientific publications
wat
Message 49213 - Posted: 27 Mar 2018 | 8:20:51 UTC - in response to Message 49212.

As you probably know, the FMA version of TN-Grid is faster than the AVX version. At least that was my result comparing a Ryzen 1700 (FMA) to an i7-4770 (AVX), both on Ubuntu.


I know.
And, i forgot, a Windows app!!! :-)

klepel
Send message
Joined: 23 Dec 09
Posts: 151
Credit: 1,972,629,934
RAC: 730,937
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49215 - Posted: 27 Mar 2018 | 17:58:25 UTC - in response to Message 49179.

The linux QM app runs fine but it requires some packages to be installed. People should expect infinite number of workunits...

We will run this for ever.

I still think, the main obstacle to gain a wider LINUX contributor base is to solve the start-up error when two WUs start at the same time. In my view, this hinders the implementation of the app on newer computers with more than 4 threads/cores, as the other available computer threads/cores are not loaded by other BOINC projects.

kain
Send message
Joined: 3 Sep 14
Posts: 120
Credit: 176,509,215
RAC: 306,573
Level
Ile
Scientific publications
watwatwatwatwat
Message 49216 - Posted: 27 Mar 2018 | 18:36:09 UTC - in response to Message 49215.

The linux QM app runs fine but it requires some packages to be installed. People should expect infinite number of workunits...

We will run this for ever.

I still think, the main obstacle to gain a wider LINUX contributor base is to solve the start-up error when two WUs start at the same time. In my view, this hinders the implementation of the app on newer computers with more than 4 threads/cores, as the other available computer threads/cores are not loaded by other BOINC projects.


True. I'm using only 2c/4t and 4c CPUs because of that. Ryzen 8c/16t is crunching WCG.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 517
Credit: 1,137,278,461
RAC: 20,414
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49217 - Posted: 27 Mar 2018 | 21:02:46 UTC
Last modified: 27 Mar 2018 | 21:45:43 UTC

You can sort of learn to live with its idiosyncrasies after a while. They are more an annoyance at first. But I wonder if a non multi-core version would fix the start-up problems? It would be worth looking into.

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 239
Credit: 550,771,414
RAC: 1,830,271
Level
Lys
Scientific publications
watwat
Message 49270 - Posted: 15 Apr 2018 | 9:16:06 UTC
Last modified: 15 Apr 2018 | 9:19:17 UTC

May I touch upon this matter again after 2,5-3 weeks... at the risk of being a pain in the neck... is there any progress regarding concurrent QC tasks? I would really like to use a 8-16 core CPU (instead of my i3) and run several jobs at the same time. Hope that the admins can take some time to follow up, in view of the many jobs we still have to crunch.

Thanks.
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

Profile Logan Carr
Send message
Joined: 12 Aug 15
Posts: 224
Credit: 41,557,936
RAC: 38,303
Level
Val
Scientific publications
wat
Message 49298 - Posted: 17 Apr 2018 | 14:10:21 UTC - in response to Message 48820.

Hey,

What are the recommended PC specs you recommend for these Linux job units? (minimal CPU and ram?) Thanks
____________
Cruncher/Learner in progress.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 517
Credit: 1,137,278,461
RAC: 20,414
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49305 - Posted: 17 Apr 2018 | 21:41:46 UTC - in response to Message 49298.

I have five QC running at the moment on two machines, and they are taking from 230 MB to 260 MB. I would plan on at least 300 MB to be safe.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 517
Credit: 1,137,278,461
RAC: 20,414
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49308 - Posted: 18 Apr 2018 | 8:26:36 UTC - in response to Message 49298.

I now have one running at 1016 MB for a few minutes, but now down to 932 MB. It looks like the upper limit is a bit elastic.

PappaLitto
Send message
Joined: 21 Mar 16
Posts: 321
Credit: 2,052,122,900
RAC: 6,368,789
Level
Phe
Scientific publications
watwat
Message 49309 - Posted: 18 Apr 2018 | 11:11:00 UTC - in response to Message 49308.

I now have one running at 1016 MB for a few minutes, but now down to 932 MB. It looks like the upper limit is a bit elastic.

Just to confirm, that's 1GB per work unit?

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 239
Credit: 550,771,414
RAC: 1,830,271
Level
Lys
Scientific publications
watwat
Message 49310 - Posted: 18 Apr 2018 | 11:18:15 UTC - in response to Message 49305.
Last modified: 18 Apr 2018 | 11:20:10 UTC

I have five QC running at the moment on two machines, and they are taking from 230 MB to 260 MB. I would plan on at least 300 MB to be safe.


pardon me for jumping in, does that mean it is possible now to run several QC tasks at the same time? For example, 4 Jobs on the Ryzen 1700?
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 638
Credit: 4,273,184
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 49311 - Posted: 18 Apr 2018 | 11:32:10 UTC - in response to Message 49310.
Last modified: 18 Apr 2018 | 11:33:45 UTC

It is possible to run as many as you want concurrently. A bug unfortunately prevents simultaneous starts. I am investigating possible workarounds but no timeline yet, sorry. Requirements are in some other thread; they are rather mild (only thing you need to have the gcc package installed).

mmonnin
Send message
Joined: 2 Jul 16
Posts: 137
Credit: 177,213,592
RAC: 512,425
Level
Ile
Scientific publications
wat
Message 49312 - Posted: 18 Apr 2018 | 11:44:27 UTC - in response to Message 49310.

I have five QC running at the moment on two machines, and they are taking from 230 MB to 260 MB. I would plan on at least 300 MB to be safe.


pardon me for jumping in, does that mean it is possible now to run several QC tasks at the same time? For example, 4 Jobs on the Ryzen 1700?


Best to just run 1 per computer via app_config or you'll eventually get two starting at once. They'll crash at once, and end up in this loop.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 517
Credit: 1,137,278,461
RAC: 20,414
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49313 - Posted: 18 Apr 2018 | 13:46:14 UTC - in response to Message 49309.

I now have one running at 1016 MB for a few minutes, but now down to 932 MB. It looks like the upper limit is a bit elastic.

Just to confirm, that's 1GB per work unit?

Yes, but now I see 1466 MB for a single work unit, the most I have seen.

Note however that I use an app_config to limit them to only one CPU core per work unit, but I do not limit the number of work units that can run at a time. It would undoubtedly be more efficient use of memory to allow the default value of four cores to run on a single work unit. It probably would not use much more memory than a single core, but what the maximum is I don't really know. I have 32 GB, so I have never really paid much attention to it. I use BOINCTasks to measure the memory use by the way.

JoergF
Avatar
Send message
Joined: 20 Apr 15
Posts: 239
Credit: 550,771,414
RAC: 1,830,271
Level
Lys
Scientific publications
watwat
Message 49314 - Posted: 18 Apr 2018 | 16:00:00 UTC - in response to Message 49311.

It is possible to run as many as you want concurrently. A bug unfortunately prevents simultaneous starts.


Okay, thank you :)
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

Profile Logan Carr
Send message
Joined: 12 Aug 15
Posts: 224
Credit: 41,557,936
RAC: 38,303
Level
Val
Scientific publications
wat
Message 49316 - Posted: 18 Apr 2018 | 21:12:02 UTC - in response to Message 49311.
Last modified: 18 Apr 2018 | 21:13:14 UTC

Requirements are in some other thread; they are rather mild (only thing you need to have the gcc package installed).


I have looked around and cannot find any requirements for this specific project. The only CPU requirements I see are here
http://www.gpugrid.net/join.php

But that looks like general use to me.

I will make my question a little more simple... would a core i3 work fine with this project, and if so, would a gen 1 be fine, gen 2, or newer? My budget for another PC dedicated to cpu work is about 100 USD. I do not mind for a small desktop either without a big gpu slot. Thank you.
____________
Cruncher/Learner in progress.

NUCCpod_NAPTIMELABS_01
Send message
Joined: 18 Aug 17
Posts: 6
Credit: 18,577,273
RAC: 406,792
Level
Pro
Scientific publications
wat
Message 49317 - Posted: 18 Apr 2018 | 22:26:57 UTC - in response to Message 49316.


I will make my question a little more simple... would a core i3 work fine with this project, and if so, would a gen 1 be fine, gen 2, or newer? My budget for another PC dedicated to cpu work is about 100 USD. I do not mind for a small desktop either without a big gpu slot. Thank you.


For questions like this, wuprop is a fantastic resource. Check the link and you can see things like time to complete for various CPUs and other requirements like memory. In this case looks like you may need 776.5 MB of RAM and an i3 should do just fine.

Profile Logan Carr
Send message
Joined: 12 Aug 15
Posts: 224
Credit: 41,557,936
RAC: 38,303
Level
Val
Scientific publications
wat
Message 49318 - Posted: 18 Apr 2018 | 23:13:18 UTC - in response to Message 49317.


I will make my question a little more simple... would a core i3 work fine with this project, and if so, would a gen 1 be fine, gen 2, or newer? My budget for another PC dedicated to cpu work is about 100 USD. I do not mind for a small desktop either without a big gpu slot. Thank you.


For questions like this, wuprop is a fantastic resource. Check the link and you can see things like time to complete for various CPUs and other requirements like memory. In this case looks like you may need 776.5 MB of RAM and an i3 should do just fine.



Thank you very much.
____________
Cruncher/Learner in progress.

Post to thread

Message boards : News : CPU jobs on Linux