Advanced search

Message boards : Graphics cards (GPUs) : Python apps for GPU hosts needs [a whole lot] more disk space.

Author Message
Jason M Sullivan
Send message
Joined: 1 Dec 21
Posts: 1
Credit: 11,545,595
RAC: 0
Level
Pro
Scientific publications
wat
Message 59121 - Posted: 17 Aug 2022 | 14:56:24 UTC

I've gotten this notice a few times over the last few months...

Python apps for GPU hosts needs 27668.09MB more disk space. You currently have 5710.52 MB available and it needs 33378.60 MB.


A little over 30Gig? That seems like an awful lot of disk to be using. I know "disk is cheap" and whatnot, but are the datasets truly that big? Anyone know why it needs so much space?

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 1284
Credit: 4,924,081,959
RAC: 6,466,138
Level
Arg
Scientific publications
watwatwatwatwat
Message 59122 - Posted: 17 Aug 2022 | 16:29:33 UTC - in response to Message 59121.

I've gotten this notice a few times over the last few months...

Python apps for GPU hosts needs 27668.09MB more disk space. You currently have 5710.52 MB available and it needs 33378.60 MB.


A little over 30Gig? That seems like an awful lot of disk to be using. I know "disk is cheap" and whatnot, but are the datasets truly that big? Anyone know why it needs so much space?

Because it DOES need that much space. The tasks do reinforcement learning using Pytorch and that framework infrastructure needs that much disk space for the examples.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59123 - Posted: 17 Aug 2022 | 16:34:00 UTC - in response to Message 59121.
Last modified: 17 Aug 2022 | 16:35:06 UTC

On Ubuntu 20.04.4, my www.gpugrid.net project directory is 3.7GB, and the slot it is running in shows 9.1GB. So that is certainly less than 30GB.

But my SSD is 250GB, and I have 170GB remaining, so I have not hit the minimum space requirements yet.

(I don't know what the maximum it needs is; it could be a lot more.)

[CSF] Aleksey Belkov
Avatar
Send message
Joined: 26 Dec 13
Posts: 85
Credit: 1,215,531,270
RAC: 182,549
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59134 - Posted: 19 Aug 2022 | 10:04:14 UTC - in response to Message 59123.

and the slot it is running in shows 9.1GB. So that is certainly less than 30GB.

Probably WUs may vary in their volume and/or occupied volume may grow during the execution of the task.
My current WU consumes almost 16 GiB(slot).

Although of course it was interesting to know exactly how the values of the required space are calculated.
Based on the fact that different values appear in such messages, this is some kind of automated algorithm, and not manually set values for WUs.

ptgst3Kfg7MDZf8ShnWCBMMnh...
Send message
Joined: 7 May 20
Posts: 2
Credit: 38,329,704
RAC: 2
Level
Val
Scientific publications
wat
Message 59180 - Posted: 1 Sep 2022 | 4:03:30 UTC
Last modified: 1 Sep 2022 | 4:09:37 UTC

deleted my post - sorry wrong thread! :)

Post to thread

Message boards : Graphics cards (GPUs) : Python apps for GPU hosts needs [a whole lot] more disk space.

//