Changes between Version 68 and Version 69 of BigCompute


Ignore:
Timestamp:
Mar 15, 2011 10:44:53 AM (7 years ago)
Author:
schaik
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • BigCompute

    v68 v69  
    118118== Workflow execution ==
    119119
    120 ''' On mini pilot. Split lines-per-file: 8,000,000. Start: 18-12-2010 16:40 '''
    121 || '''Lane''' || '''split BWA merge''' || '''Comments''' || '''Elapsed time (s)''' ||
    122 || A4a_L4_HUModqRBUDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-cOunUZ/html/workflow-cOunUZ.html done] || || 22740 ||
    123 || A4a_L6_HUModqRADDIAAPE  || [https://hangplek.nikhef.nl:443/workflows/workflow-pqWdc1/html/workflow-pqWdc1.html done] || || 37750 ||
    124 || A4a_L7_HUModqRBVDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-OqeNdv/html/workflow-OqeNdv.html failed] || || ||
    125 || A4b_L3_HUModqRAFDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-GRDX0X/html/workflow-GRDX0X.html done] || || 32256 ||
    126 || A4b_L6_HUModqRBTDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-ZRH98o/html/workflow-ZRH98o.html failed] || || ||
    127 || R2A _L1_HUModqRADDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-1lNDQh/html/workflow-1lNDQh.html failed] || || ||
    128 || R2A _L1_HUModqRAFDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-rufel6/html/workflow-rufel6.html done] || || 35819 ||
    129 || R2A _L5_HUModqRAEDIAAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-wVuMjg/html/workflow-wVuMjg.html failed] || || ||
    130 || R2B _L3_HUModqRBTDIAAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-L3m5Yi/html/workflow-L3m5Yi.html done] || || 23754 ||
    131 || R2B _L4_HUModqRBUDIAAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-alGMJK/html/workflow-alGMJK.html failed] || pair 1 not in correct gzip format? || ||
    132 || R2B _L6_HUModqRBTDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-LCaDkY/html/workflow-LCaDkY.html failed] || || ||
    133 || R2C _L2_HUModqRBUDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-gVXW9i/html/workflow-gVXW9i.html done] || || 40763 ||
    134 || R2C _L2_HUModqRBVDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-8bma4j/html/workflow-8bma4j.html failed] || || ||
    135 || R2C _L7_HUModqRBVDIAAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-Rjh1lC/html/workflow-Rjh1lC.html done] || || 39374 ||
    136 || Unknown_L6_HUModqRBUDIAAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-IG0HAm/html/workflow-IG0HAm.html done] || || 30307 ||
    137 Note: The Merge component is red (marked as failed) in the finished workflows. The bam files where produced successfully.
     120|| '''#''' || '''Sample''' || '''WF''' || '''Status''' || '''Start''' ||
     121|| 1 || A4a || [http://orange.ebioscience.amc.nl/workflows/workflow-693426f3/html/workflow-693426f3.html F] [http://orange.ebioscience.amc.nl/workflows/workflow-35a9b777/html/workflow-35a9b777.html F] [http://orange.ebioscience.amc.nl/workflows/workflow-4ba3f651/html/workflow-4ba3f651.html F] [http://orange.ebioscience.amc.nl/workflows/workflow-425d9ceb/html/workflow-425d9ceb.html workflow-425d9ceb] || done ||||
     122|| 2 || Vartest || [http://orange.ebioscience.amc.nl/workflows/workflow-490b15f8/html/workflow-490b15f8.html workflow-490b15f8] || done ||||
     123|| 3 || Iteration test || [http://orange.ebioscience.amc.nl/workflows/workflow-bf48aff1/html/workflow-bf48aff1.html workflow-bf48aff1] || failed ||||
     124|| 4 || Iteration test || [http://orange.ebioscience.amc.nl/workflows/workflow-923c6588/html/workflow-923c6588.html workflow-923c6588] || done ||||
     125|| 5 || 60-samples-batch (15 lanes) || [http://orange.ebioscience.amc.nl/workflows/workflow-d80b5767/html/workflow-d80b5767.html workflow-d80b5767] || 10 / 15 done || 11-02-2011 19:30 ||
     126|| 6 || 60-samples-batch A (55 lanes) || [http://orange.ebioscience.amc.nl/workflows/workflow-cbaca6e5/html/workflow-cbaca6e5.html workflow-cbaca6e5] || 15 / 55 done || 12-02-2011 13:55 ||
     127|| 7 || 60-samples-batch A remaining 1 (17 lanes) || [http://orange.ebioscience.amc.nl/workflows/workflow-835250b1/html/workflow-835250b1.html workflow-835250b1] || failed (grid very busy) || 07-03-2011 17:45 ||
     128|| 8 || 60-samples-batch A remaining 1 (17 lanes) || [http://orange.ebioscience.amc.nl/workflows/workflow-31eb952d/html/workflow-31eb952d.html workflow-31eb952d] || 1/17 done || 08-03-2011 14:15 ||
     129|| 9 || 60-samples-batch A remaining (27 lanes) || [http://orange.ebioscience.amc.nl/workflows/workflow-fd98c7c8/html/workflow-fd98c7c8.html workflow-fd98c7c8] || running || 15-03-2011 10:45 ||
     130|| x || 60-samples-batch G (54 lanes) || || || ||
     131|| x || 60-samples-batch R (60 lanes) || || || ||
    138132
    139 ''' BWA alignment on mini pilot without splitting the data '''
    140 * 19-12-2010 14:00 [https://hangplek.nikhef.nl:443/workflows/workflow-rIUQ2N/html/workflow-rIUQ2N.html done] - elapsed time 325886 s
    141 
    142 '''PicardQC on finished bam files'''
    143 * 20-12-2010 11:30 [https://hangplek.nikhef.nl:443/workflows/workflow-3T5uMC/html/workflow-3T5uMC.html done] - elapsed time (10 bam files) 8730 s
    144 * [http://www.bbmriwiki.nl/attachment/wiki/BigCompute/log-picardqc-20101220.ods Info runtime and used disk space] (ods)
    145 
    146 '''Coverage-per-base on finished bam files'''
    147 * 20-12-2010 12:40 [https://hangplek.nikhef.nl:443/workflows/workflow-qwMQka/html/workflow-qwMQka.html done] - elapsed time 89634 s
    148 
    149 '''Mark-duplicates on finished bam files'''
    150 * 20-12-2010 12:47 [https://hangplek.nikhef.nl:443/workflows/workflow-E6tgrW/html/workflow-E6tgrW.html done] - elapsed time 10614 s
    151 
    152 '''Re-align on finished bam files'''
    153 * 20-12-2010 17:48 [https://hangplek.nikhef.nl:443/workflows/workflow-ICmgZF/html/workflow-ICmgZF.html done] - elapsed time 9165 s
    154 
    155 '''Gvnl pipeline on remaining samples (split, bwa, merge, coverage-per-base, picard-qc, mark-duplicates, re-align) Start: 20-12-2010 18:50'''
    156 || '''Lane''' || '''Progress''' || '''Elapsed time''' ||
    157 || A4a_L7_HUModqRBVDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-RllyLv/html/workflow-RllyLv.html done] || 208542 ||
    158 || A4b_L6_HUModqRBTDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-A6WSPB/html/workflow-A6WSPB.html done] || 74476 ||
    159 || R2A _L1_HUModqRADDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-L2NZUb/html/workflow-L2NZUb.html done] || 72636 ||
    160 || R2A _L5_HUModqRAEDIAAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-VoZud6/html/workflow-VoZud6.html done] || 55242 ||
    161 || R2B _L4_HUModqRBUDIAAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-ldqezK/html/workflow-ldqezK.html failed] || ||
    162 || R2B _L6_HUModqRBTDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-q9PKaQ/html/workflow-q9PKaQ.html done] || 227859 ||
    163 || R2C _L2_HUModqRBVDIBAPE || [https://hangplek.nikhef.nl:443/workflows/workflow-jroCmW/html/workflow-jroCmW.html done] || 44730 ||
    164 * Note1: Workflow stops after merge-bam. Working on it.
    165 * Note2: The Dutch grid seems very busy at the moment. That could explain the longer execution times.
    166 
    167 '''Gvnl pipeline on theoretical reads'''
    168   * 22-12-2010 19:14 [https://hangplek.nikhef.nl/workflows//workflow-FRErry/html/workflow-FRErry.html done] - elapsed time 38587 s
    169   * Workflow does continue after the merge-bam now
    170   * To do: implement recalibration step, and a check on all bam files of the split input before the merge step
    171133
    172134== Alternatives ==