recon_2019-01_ver02_batch01


Click on each figure to show larger version

Status

Info retrieved: 2019-10-22 14:45:35.0

Job Limit: 0 Total attempts: 3579

Status from SWIF Summary

Numbers are for individual registered jobs, final status.

Undispatched Dispatched Succeeded Failed Problems Canceled Total
0 10 3442 0 17 0 3469


Auger Status

Dependency Pending Staging In Active Staging Out Finishing Dispatched Total
0 0 0 10 0 0 10

Status by Resources

Counts are for each attempt of running a job, if an attempt fails and the job is resubmitted

the same job will appear in other places as other attempts.

Resources Results TOTAL
RAM (GB) time limit (hrs) SUCCESS USER ERROR FAILED TIMEOUT OVER_RLIMIT UNDISPATCHED ACTIVE CANCELLED
12.0 24.0 3423 40 1 54 2 0 7 0 3527
12.0 30.0 19 15 0 0 1 0 3 0 38
12.0 36.0 0 14 0 0 0 0 0 0 14
TOTAL 3442 69 1 54 3 0 10 0 3579

Number of Attempts For Each Job


Number of jobs in each stage since launch


Time Spent in Dependency


Time Spent in Pending


Number of Jobs in Each Stage


Duration of Each Stage


Wall Time, CPU Time, CPU Time vs Wall Time


MAX Memory reported by AUGER


Problem Jobs

Run File RAM (GB) Attempt Problem Resolution SWIF Job ID Auger Job ID Submit Time Complete Time
060708 003 12 1 / 2 AUGER-TIMEOUT retry 23742227 5674526 2019-10-16 12:00:31.0 2019-10-18 08:00:03.0
060715 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23742253 5674552 2019-10-16 12:00:33.0 2019-10-17 11:23:07.0
060715 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23742253 5808742 2019-10-21 02:41:01.0 2019-10-21 16:23:40.0
060715 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23742253 5822410 2019-10-21 18:11:38.0 2019-10-21 19:18:59.0
060715 000 12 4 / 4 SWIF-USER-NON-ZERO 23742253 5835118 2019-10-22 09:56:34.0 2019-10-22 10:00:11.0
060727 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23742497 5674796 2019-10-16 12:00:51.0 2019-10-16 12:06:18.0
060727 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23742497 5808743 2019-10-21 02:41:01.0 2019-10-21 16:21:11.0
060727 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23742497 5822411 2019-10-21 18:11:38.0 2019-10-21 19:19:12.0
060727 000 12 4 / 4 SWIF-USER-NON-ZERO 23742497 5835119 2019-10-22 09:56:34.0 2019-10-22 10:00:11.0
060732 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23742645 5674944 2019-10-16 12:01:08.0 2019-10-16 12:06:34.0
060732 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23742645 5808744 2019-10-21 02:41:01.0 2019-10-21 16:28:39.0
060732 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23742645 5822412 2019-10-21 18:11:38.0 2019-10-21 19:19:21.0
060732 000 12 4 / 4 SWIF-USER-NON-ZERO 23742645 5835120 2019-10-22 09:56:34.0 2019-10-22 10:00:11.0
060749 000 12 1 / 2 AUGER-TIMEOUT retry 23742956 5675255 2019-10-16 12:01:31.0 2019-10-18 17:09:05.0
060751 007 12 1 / 3 SWIF-USER-NON-ZERO retry 23742983 5675282 2019-10-16 12:01:33.0 2019-10-19 06:31:37.0
060751 007 12 2 / 3 SWIF-USER-NON-ZERO modify 23742983 5808746 2019-10-21 02:41:01.0 2019-10-21 22:44:37.0
060753 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23743035 5675334 2019-10-16 12:01:37.0 2019-10-17 08:20:06.0
060753 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23743035 5808747 2019-10-21 02:41:01.0 2019-10-21 16:21:28.0
060753 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23743035 5822413 2019-10-21 18:11:38.0 2019-10-21 19:19:24.0
060753 000 12 4 / 4 SWIF-USER-NON-ZERO 23743035 5835121 2019-10-22 09:56:34.0 2019-10-22 10:00:11.0
060755 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23743089 5675388 2019-10-16 12:01:41.0 2019-10-17 17:35:55.0
060755 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23743089 5808748 2019-10-21 02:41:02.0 2019-10-21 16:24:00.0
060755 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23743089 5822414 2019-10-21 18:11:38.0 2019-10-21 19:20:06.0
060755 000 12 4 / 4 SWIF-USER-NON-ZERO 23743089 5835122 2019-10-22 09:56:34.0 2019-10-22 10:00:47.0
060756 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23743090 5675389 2019-10-16 12:01:41.0 2019-10-17 17:36:45.0
060756 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23743090 5808749 2019-10-21 02:41:02.0 2019-10-21 16:24:12.0
060756 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23743090 5822415 2019-10-21 18:11:38.0 2019-10-21 19:20:04.0
060756 000 12 4 / 4 SWIF-USER-NON-ZERO 23743090 5835123 2019-10-22 09:56:34.0 2019-10-22 10:00:45.0
060757 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23743091 5675390 2019-10-16 12:01:41.0 2019-10-17 17:37:22.0
060757 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23743091 5808750 2019-10-21 02:41:02.0 2019-10-21 16:24:15.0
060757 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23743091 5822416 2019-10-21 18:11:39.0 2019-10-21 19:20:08.0
060757 000 12 4 / 4 SWIF-USER-NON-ZERO 23743091 5835124 2019-10-22 09:56:34.0 2019-10-22 10:00:29.0
060761 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23743097 5675396 2019-10-16 12:01:41.0 2019-10-17 18:03:08.0
060761 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23743097 5808751 2019-10-21 02:41:02.0 2019-10-21 16:22:03.0
060761 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23743097 5822417 2019-10-21 18:11:39.0 2019-10-21 19:20:06.0
060761 000 12 4 / 4 SWIF-USER-NON-ZERO 23743097 5835125 2019-10-22 09:56:35.0 2019-10-22 10:00:11.0
060775 000 12 1 / 2 AUGER-TIMEOUT retry 23743358 5675657 2019-10-16 12:02:02.0 2019-10-19 19:33:05.0
060776 007 12 1 / 3 AUGER-OVER_RLIMIT retry 23743366 5675665 2019-10-16 12:02:02.0 2019-10-19 02:05:26.0
060776 007 12 2 / 3 AUGER-OVER_RLIMIT modify 23743366 5808753 2019-10-21 02:41:02.0 2019-10-22 03:47:28.0
060776 007 12 3 / 3 AUGER-OVER_RLIMIT 23743366 5835133 2019-10-22 09:56:35.0 2019-10-22 13:33:14.0
060779 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23743385 5675684 2019-10-16 12:02:04.0 2019-10-18 15:47:29.0
060779 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23743385 5808754 2019-10-21 02:41:02.0 2019-10-21 16:21:59.0
060779 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23743385 5822418 2019-10-21 18:11:39.0 2019-10-21 19:20:14.0
060779 000 12 4 / 4 SWIF-USER-NON-ZERO 23743385 5835126 2019-10-22 09:56:35.0 2019-10-22 10:00:11.0
060790 011 12 1 / 2 SWIF-USER-NON-ZERO retry 23743831 5676130 2019-10-16 12:02:39.0 2019-10-17 08:13:53.0
060790 018 12 1 / 2 SWIF-USER-NON-ZERO retry 23743838 5676137 2019-10-16 12:02:40.0 2019-10-17 08:56:51.0
060791 023 12 1 / 2 SWIF-USER-NON-ZERO retry 23743899 5676198 2019-10-16 12:02:44.0 2019-10-17 23:11:36.0
060796 016 12 1 / 2 SWIF-USER-NON-ZERO retry 23744219 5676518 2019-10-16 12:03:10.0 2019-10-18 03:01:09.0
060798 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23744334 5676663 2019-10-16 12:03:17.0 2019-10-20 09:59:26.0
060798 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23744334 5808759 2019-10-21 02:41:02.0 2019-10-21 16:23:43.0
060798 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23744334 5822419 2019-10-21 18:11:39.0 2019-10-21 19:20:15.0
060798 000 12 4 / 4 SWIF-USER-NON-ZERO 23744334 5835127 2019-10-22 09:56:35.0 2019-10-22 10:00:11.0
060798 060 12 1 / 3 SWIF-USER-NON-ZERO retry 23744394 5676771 2019-10-16 12:03:21.0 2019-10-18 09:22:00.0
060798 060 12 2 / 3 SWIF-USER-NON-ZERO modify 23744394 5808760 2019-10-21 02:41:02.0 2019-10-21 20:44:36.0
060798 060 12 3 / 3 SWIF-USER-NON-ZERO 23744394 5835134 2019-10-22 09:56:36.0 2019-10-22 13:27:03.0
060802 067 12 1 / 2 AUGER-TIMEOUT retry 23744588 5677087 2019-10-16 12:03:34.0 2019-10-19 01:51:54.0
060807 000 12 1 / 2 AUGER-TIMEOUT retry 23744773 5677372 2019-10-16 12:04:09.0 2019-10-20 10:19:30.0
060807 001 12 1 / 2 AUGER-TIMEOUT retry 23744774 5677373 2019-10-16 12:04:09.0 2019-10-20 10:21:00.0
060808 000 12 1 / 2 AUGER-TIMEOUT retry 23744775 5677374 2019-10-16 12:04:09.0 2019-10-20 10:34:33.0
060808 001 12 1 / 2 AUGER-TIMEOUT modify 23744776 5677375 2019-10-16 12:04:09.0 2019-10-21 18:51:55.0
060809 000 12 1 / 2 AUGER-TIMEOUT retry 23744777 5677376 2019-10-16 12:04:09.0 2019-10-20 10:35:03.0
060811 031 12 1 / 2 AUGER-TIMEOUT retry 23744822 5677421 2019-10-16 12:04:12.0 2019-10-19 04:03:55.0
060811 042 12 1 / 2 AUGER-TIMEOUT retry 23744835 5677434 2019-10-16 12:04:13.0 2019-10-19 22:12:55.0
060811 037 12 1 / 2 AUGER-TIMEOUT retry 23744836 5677435 2019-10-16 12:04:13.0 2019-10-20 11:53:24.0
060812 011 12 1 / 2 AUGER-TIMEOUT modify 23744867 5677466 2019-10-16 12:04:15.0 2019-10-21 12:13:00.0
060812 014 12 1 / 2 AUGER-TIMEOUT retry 23744870 5677469 2019-10-16 12:04:15.0 2019-10-20 12:08:02.0
060815 017 12 1 / 2 AUGER-TIMEOUT retry 23744943 5677542 2019-10-16 12:04:20.0 2019-10-19 02:09:26.0
060815 033 12 1 / 2 AUGER-TIMEOUT retry 23744964 5677563 2019-10-16 12:04:21.0 2019-10-19 02:12:28.0
060815 041 12 1 / 2 AUGER-TIMEOUT retry 23744966 5677565 2019-10-16 12:04:21.0 2019-10-19 02:17:27.0
060815 053 12 1 / 2 AUGER-TIMEOUT retry 23744979 5677578 2019-10-16 12:04:22.0 2019-10-19 02:24:29.0
060816 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23744984 5677583 2019-10-16 12:04:22.0 2019-10-17 08:21:14.0
060816 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23744984 5808774 2019-10-21 02:41:04.0 2019-10-21 16:25:16.0
060816 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23744984 5822421 2019-10-21 18:11:39.0 2019-10-21 19:20:17.0
060816 000 12 4 / 4 SWIF-USER-NON-ZERO 23744984 5835128 2019-10-22 09:56:35.0 2019-10-22 10:00:11.0
060817 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23744985 5677584 2019-10-16 12:04:22.0 2019-10-18 01:38:18.0
060817 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23744985 5808775 2019-10-21 02:41:04.0 2019-10-21 16:28:36.0
060817 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23744985 5822422 2019-10-21 18:11:39.0 2019-10-21 19:20:18.0
060817 000 12 4 / 4 SWIF-USER-NON-ZERO 23744985 5835129 2019-10-22 09:56:35.0 2019-10-22 09:59:02.0
060818 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23744986 5677585 2019-10-16 12:04:22.0 2019-10-18 01:38:30.0
060818 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23744986 5808776 2019-10-21 02:41:04.0 2019-10-21 16:29:06.0
060818 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23744986 5822423 2019-10-21 18:11:39.0 2019-10-21 19:20:23.0
060818 000 12 4 / 4 SWIF-USER-NON-ZERO 23744986 5835130 2019-10-22 09:56:35.0 2019-10-22 10:00:12.0
060819 006 12 1 / 2 AUGER-TIMEOUT retry 23744992 5677591 2019-10-16 12:04:23.0 2019-10-20 07:10:01.0
060819 009 12 1 / 2 AUGER-TIMEOUT retry 23744997 5677596 2019-10-16 12:04:23.0 2019-10-19 02:38:34.0
060819 023 12 1 / 2 AUGER-TIMEOUT retry 23745009 5677608 2019-10-16 12:04:24.0 2019-10-18 10:58:40.0
060819 023 12 2 / 2 SWIF-USER-NON-ZERO 23745009 5808779 2019-10-21 02:41:04.0 2019-10-22 14:11:13.0
060819 024 12 1 / 2 AUGER-TIMEOUT retry 23745010 5677609 2019-10-16 12:04:24.0 2019-10-18 10:34:36.0
060819 026 12 1 / 2 AUGER-TIMEOUT retry 23745014 5677613 2019-10-16 12:04:24.0 2019-10-18 11:00:15.0
060819 035 12 1 / 2 AUGER-TIMEOUT retry 23745022 5677621 2019-10-16 12:04:25.0 2019-10-18 10:25:36.0
060819 039 12 1 / 2 AUGER-TIMEOUT retry 23745024 5677623 2019-10-16 12:04:25.0 2019-10-19 02:24:29.0
060819 045 12 1 / 2 AUGER-TIMEOUT retry 23745035 5677634 2019-10-16 12:04:26.0 2019-10-20 04:07:08.0
060821 001 12 1 / 2 AUGER-TIMEOUT retry 23745070 5677669 2019-10-16 12:04:28.0 2019-10-18 11:12:40.0
060822 005 12 1 / 2 AUGER-TIMEOUT retry 23745107 5677706 2019-10-16 12:04:31.0 2019-10-18 11:17:13.0
060822 009 12 1 / 2 AUGER-TIMEOUT retry 23745109 5677708 2019-10-16 12:04:31.0 2019-10-18 10:17:36.0
060822 033 12 1 / 2 AUGER-TIMEOUT retry 23745125 5677724 2019-10-16 12:04:34.0 2019-10-20 04:18:41.0
060822 031 12 1 / 2 AUGER-TIMEOUT retry 23745137 5677736 2019-10-16 12:04:35.0 2019-10-19 04:11:55.0
060822 041 12 1 / 2 AUGER-TIMEOUT retry 23745143 5677742 2019-10-16 12:04:36.0 2019-10-19 04:37:00.0
060822 049 12 1 / 2 AUGER-TIMEOUT retry 23745152 5677751 2019-10-16 12:04:36.0 2019-10-20 12:27:36.0
060823 011 12 1 / 2 AUGER-TIMEOUT retry 23745172 5677771 2019-10-16 12:04:37.0 2019-10-19 04:24:58.0
060823 019 12 1 / 2 AUGER-TIMEOUT retry 23745179 5677778 2019-10-16 12:04:38.0 2019-10-19 04:44:30.0
060823 026 12 1 / 2 AUGER-TIMEOUT retry 23745188 5677787 2019-10-16 12:04:38.0 2019-10-19 04:45:00.0
060823 051 12 1 / 3 SWIF-USER-NON-ZERO retry 23745212 5677811 2019-10-16 12:04:40.0 2019-10-18 18:46:01.0
060823 051 12 2 / 3 SWIF-USER-NON-ZERO modify 23745212 5808795 2019-10-21 02:41:05.0 2019-10-22 03:32:13.0
060825 049 12 1 / 2 AUGER-TIMEOUT modify 23745282 5677881 2019-10-16 12:04:44.0 2019-10-21 10:58:51.0
060825 050 12 1 / 2 AUGER-TIMEOUT modify 23745283 5677882 2019-10-16 12:04:44.0 2019-10-21 11:00:20.0
060826 001 12 1 / 2 AUGER-TIMEOUT modify 23745294 5677893 2019-10-16 12:04:45.0 2019-10-21 11:05:52.0
060826 008 12 1 / 2 AUGER-TIMEOUT modify 23745302 5677901 2019-10-16 12:04:45.0 2019-10-21 11:57:57.0
060826 009 12 1 / 2 AUGER-TIMEOUT modify 23745303 5677902 2019-10-16 12:04:45.0 2019-10-21 11:58:57.0
060827 002 12 1 / 2 SWIF-USER-NON-ZERO retry 23745319 5677918 2019-10-16 12:04:46.0 2019-10-20 11:58:50.0
060827 013 12 1 / 2 AUGER-TIMEOUT modify 23745330 5677929 2019-10-16 12:04:47.0 2019-10-21 12:04:28.0
060827 015 12 1 / 2 AUGER-TIMEOUT modify 23745332 5677931 2019-10-16 12:04:47.0 2019-10-21 12:08:29.0
060828 002 12 1 / 2 AUGER-TIMEOUT modify 23745383 5677982 2019-10-16 12:04:51.0 2019-10-21 12:13:00.0
060828 005 12 1 / 2 AUGER-TIMEOUT modify 23745386 5677985 2019-10-16 12:04:51.0 2019-10-21 12:13:29.0
060828 010 12 1 / 2 AUGER-TIMEOUT modify 23745391 5677990 2019-10-16 12:04:51.0 2019-10-21 12:22:00.0
060828 015 12 1 / 2 AUGER-TIMEOUT modify 23745396 5677995 2019-10-16 12:04:51.0 2019-10-21 12:28:02.0
060828 017 12 1 / 2 AUGER-TIMEOUT modify 23745398 5677997 2019-10-16 12:04:51.0 2019-10-21 12:30:34.0
060828 020 12 1 / 2 AUGER-TIMEOUT modify 23745401 5678000 2019-10-16 12:04:52.0 2019-10-21 13:02:11.0
060828 030 12 1 / 2 AUGER-TIMEOUT modify 23745411 5678010 2019-10-16 12:04:52.0 2019-10-21 13:51:25.0
060829 004 12 1 / 2 AUGER-TIMEOUT modify 23745421 5678020 2019-10-16 12:04:53.0 2019-10-21 12:58:08.0
060829 005 12 1 / 2 AUGER-TIMEOUT modify 23745422 5678021 2019-10-16 12:04:53.0 2019-10-21 12:58:07.0
060829 007 12 1 / 2 AUGER-TIMEOUT modify 23745424 5678023 2019-10-16 12:04:53.0 2019-10-21 12:44:04.0
060829 008 12 1 / 2 AUGER-TIMEOUT modify 23745425 5678024 2019-10-16 12:04:53.0 2019-10-21 12:50:37.0
060830 000 12 1 / 4 SWIF-USER-NON-ZERO retry 23745428 5678027 2019-10-16 12:04:53.0 2019-10-20 12:12:48.0
060830 000 12 2 / 4 SWIF-USER-NON-ZERO modify 23745428 5808797 2019-10-21 02:41:05.0 2019-10-21 16:30:47.0
060830 000 12 3 / 4 SWIF-USER-NON-ZERO modify 23745428 5822442 2019-10-21 18:11:40.0 2019-10-21 19:27:09.0
060830 000 12 4 / 4 SWIF-USER-NON-ZERO 23745428 5835131 2019-10-22 09:56:35.0 2019-10-22 10:00:11.0
060833 008 12 1 / 2 AUGER-FAILED retry 23745530 5678129 2019-10-16 12:05:00.0 2019-10-21 00:55:56.0