Author |
Message |
|
Noticed Upload failures - starting little over an hour ago
Kind Regards,
BER
3/5/2016 6:54:29 PM | GPUGRID | [error] Error reported by file upload server: can't write file /home/ps3grid/projects/PS3GRID/upload/279/002_640_902x1x48-GERARD_CXCL12_CostaMonomerx_umbrella-0-1-RND9106_0_9: No space left on server
3/5/2016 6:54:29 PM | GPUGRID | Temporarily failed upload of 002_640_902x1x48-GERARD_CXCL12_CostaMonomerx_umbrella-0-1-RND9106_0_9: transient upload error
3/5/2016 6:54:29 PM | GPUGRID | Backing off 00:03:29 on upload of 002_640_902x1x48-GERARD_CXCL12_CostaMonomerx_umbrella-0-1-RND9106_0_9
3/5/2016 6:57:59 PM | GPUGRID | Started upload of 002_640_902x1x48-GERARD_CXCL12_CostaMonomerx_umbrella-0-1-RND9106_0_9
3/5/2016 6:58:01 PM | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
3/5/2016 6:58:01 PM | GPUGRID | Temporarily failed upload of 002_640_902x1x48-GERARD_CXCL12_CostaMonomerx_umbrella-0-1-RND9106_0_9: transient upload error
|
|
|
|
Still receiving this message |
|
|
|
I encountered this error first this morning - and throughout the day.
3/6/2016 6:45:01 AM | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
3/6/2016 8:20:54 PM | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
|
|
|
|
The problem lasts for 10 hours already and one cannot return the results.
| GPUGRID | [error] Error reported by file upload server: Server is out of disk space
|
|
|
|
only three days left to fix this problem:
07.03.2016 08:37:23 | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
07.03.2016 08:37:23 | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
I was just preparing for this team challenge:
CRUNCHATHLON GPUGRID [MONTHLY RUN - RACE XIX]
Start time: 2016-03-10 18:00 UTC
End time: 2016-03-14 18:00 UTC[/url] |
|
|
|
We are now trying to fix this issue. Sorry for the inconvinience! |
|
|
|
We are now trying to fix this issue. Sorry for the inconvinience!
That worked brilliantly to clear the backlog - thanks! - but I'm afraid we've filled it up again... |
|
|
|
I am seeing this in my event log in addition to the out of space messages:
3/7/2016 7:44:07 AM | GPUGRID | Sending scheduler request: To fetch work.
3/7/2016 7:44:07 AM | GPUGRID | Requesting new tasks for NVIDIA GPU
3/7/2016 7:44:09 AM | GPUGRID | Scheduler request completed: got 0 new tasks
3/7/2016 7:44:09 AM | GPUGRID | No tasks sent
3/7/2016 7:44:09 AM | GPUGRID | This computer has reached a limit on tasks in progress
|
|
|
WPrionSend message
Joined: 30 Apr 13 Posts: 87 Credit: 1,065,409,111 RAC: 0 Level
Scientific publications
|
Same problems since yesterday. On the bright side it's great news for Asteroids and SETI! ;-)
ETA:
3/6/2016 11:51:40 AM | GPUGRID | Sending scheduler request: Requested by project.
3/6/2016 11:51:40 AM | GPUGRID | Requesting new tasks for NVIDIA GPU
3/6/2016 11:51:42 AM | GPUGRID | Scheduler request completed: got 0 new tasks
3/6/2016 11:51:42 AM | GPUGRID | No tasks sent
3/6/2016 11:51:42 AM | GPUGRID | This computer has reached a limit on tasks in progress
3/6/2016 12:09:06 PM | GPUGRID | Computation for task e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0 finished
3/6/2016 12:09:19 PM | GPUGRID | Started upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_0
3/6/2016 12:09:19 PM | GPUGRID | Started upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_1
3/6/2016 12:09:20 PM | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
3/6/2016 12:09:20 PM | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
3/6/2016 12:09:20 PM | GPUGRID | Temporarily failed upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_0: transient upload error
3/6/2016 12:09:20 PM | GPUGRID | Backing off 00:02:33 on upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_0
3/6/2016 12:09:20 PM | GPUGRID | Temporarily failed upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_1: transient upload error
3/6/2016 12:09:20 PM | GPUGRID | Backing off 00:02:09 on upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_1
3/6/2016 12:09:20 PM | GPUGRID | Started upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_2
3/6/2016 12:09:20 PM | GPUGRID | Started upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_3
3/6/2016 12:09:21 PM | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
3/6/2016 12:09:21 PM | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
3/6/2016 12:09:21 PM | GPUGRID | Temporarily failed upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_2: transient upload error
3/6/2016 12:09:21 PM | GPUGRID | Backing off 00:03:27 on upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_2
3/6/2016 12:09:21 PM | GPUGRID | Temporarily failed upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_3: transient upload error
3/6/2016 12:09:21 PM | GPUGRID | Backing off 00:03:28 on upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_3
3/6/2016 12:09:21 PM | GPUGRID | Started upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_7
3/6/2016 12:09:21 PM | GPUGRID | Started upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_9
3/6/2016 12:09:23 PM | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
3/6/2016 12:09:23 PM | GPUGRID | Finished upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_7
3/6/2016 12:09:23 PM | GPUGRID | Temporarily failed upload of e1s9_1-GERARD_A2AR_luf6632_a_1Dx1-1-2-RND8793_0_9: transient upload error
3/6/2016 12:09:23 PM | GPUGRID | Backing off 00:02:24 on up |
|
|
|
No changes yet. Continuously same stupid message "out of disk space" :-(
O.K., we try to be patient. |
|
|
WPrionSend message
Joined: 30 Apr 13 Posts: 87 Credit: 1,065,409,111 RAC: 0 Level
Scientific publications
|
I now have an active upload! Yeah!!! I really didn't want to dump a day's work to get new tasks.
Thanks!!
W
EDIT - it's back in retry mode. The largest file reports 100% uploaded, but still didn't clear from the queue. The retry delays for the remaining files range between 1 and 5 hours. :-( |
|
|
|
Same here! Great! |
|
|
|
And I've just been able to clear half-a-dozen of the tasks I received following this morning's 'clearance'. But it still feels as if the storage server is teetering on a knife-edge - I take it the staff are scouring the local corner-shops for fresh hard disks and memory sticks? ;-) |
|
|
TJSend message
Joined: 26 Jun 09 Posts: 815 Credit: 1,470,385,294 RAC: 0 Level
Scientific publications
|
Uploading just ready but was not able since yesterday, also no new work. And sites is slow. But all was a result of the 7000+ short runs off course that where ran with the power horses that doe the long runs normally so a lot of server traffic. This will happen again if the same situation occurs.
P.S. downloading new work goes very slow at the moment.
Happy crunching
____________
Greetings from TJ |
|
|
|
I am freeing more space now. Seems that we overloaded the server! :D |
|
|