View source for CAGEscan

== Introduction ==

CAGEscan is a technology where paired-ends of random-primed 5'-capped molecules are sequenced, thus associating TSS to collections of downstream exonic sequences scanning the transcript.  For a more complete description of CAGEscan and its potential application, please refer to:

'''Linking promoters to functional transcripts in small samples with nanoCAGE and CAGEscan'''. ''[http://www.nature.com/nmeth/journal/v7/n7/full/nmeth.1470.html Nat Methods. 2010 Jul;7(7):528-34]''. Plessy C, Bertin N, Takahashi H, Simone R, Salimullah M, Lassmann T, Vitezic M, Severin J, Olivarius S, Lazarevic D, Hornig N, Orlando V, Bell I, Gao H, Dumais J, Kapranov P, Wang H, Davis CA, Gingeras TR, Kawai J, Daub CO, Hayashizaki Y, Gustincich S, Carninci P. [[Media:Plessy_NatMeth2010.cagescan.pdf|(PDF)]] 

FANTOM5 CAGEscan libraries were be produced from biological samples selected for the diversity of transcripts they produced.

The primary purpose of those libraries is to associate HelicosCAGE derived TSS (in particular orphan DPI clusters) to annotations such as known transcripts, and in last ressort to provide a partial reconstitution of the intron/exon structure of the TSS product.

== Protocols  ==

*CAGEscan library construction protocol: [[OP-SOLEXA-nanoCAGE-Direct-v1.4]]
*CAGEscan mapping protocol: [[CAGEscan_mapping_protocol]] 
*CAGEscan-driven HelicosCAGE TSS&nbsp;annotation: [[CAGEscan_driven_annotation_protocol]] 
*CAGEscan clustering protocol: [[CAGEscan_clustering_protocol]]

== Datasets and links ==

* RAW data and aligned reads: the subdirectories marked ''CAGEscan'' in https://fantom5-collaboration.gsc.riken.jp/files/data/shared/UPDATE_023/f5pipeline/

* LS-archive IDs: (search for "''FANTOM5 CAGEScan''") 841 (pilot) 861~866 (runs), 961~966 (re-sequenced lanes).

* Library IDs: NCig10013-NCig10068, NCig10126-NCig10131

<pre>
ls /analysisdata/mirrors/F5_file_current/UPDATE_024/f5pipeline/*/*NCig*.5prime.fq.gz | grep -o NCig..... | sort -u
</pre>

* DDBJ Sequence Read Archive, DRA005606 (2017).

=== First version of the alignments (until UPDATE_025 incl.) ===

'''Not used in the Data Descriptor.'''

* Original [[CAGEscan_mapping_protocol]] (`pairedBamToBed12`, `CAGEscan-Clustering.pl`).
* Reads and alignments: https://fantom5-collaboration.gsc.riken.jp/files/data/shared/UPDATE_026/f5pipeline/.
* CAGEscan clusters seeded with FANTOM5 DPI clusters: https://fantom5-collaboration.gsc.riken.jp/webdav/home/nbertin/CAGEscan-FREEZE_Phase1_2_1pls2-Cluster/.

(''old links follow'')

* [[File:CAGEscan_production_schedule_February_2011.xls]] (Delayed by earthquake).
* [[File:CAGEscan_production_schedule_April04_2012.xls]]

*Biological sample / library list and direct relation to HelicosCAGE libraries [https://fantom5-collaboration.gsc.riken.jp/webdav/home/nbertin/CAGEscan-Data-UPDATE_July11/library_id.01Jul11.txt CAGEscan-Data-UPDATE_July11 libraries listing]

*[https://fantom5-collaboration.gsc.riken.jp/webdav/home/nbertin/CAGEscan-Data-UPDATE_July11/ SAM/BAM formatted paired-end mapping files]
*[https://fantom5-collaboration.gsc.riken.jp/webdav/home/nbertin/CAGEscan-Data-UPDATE_July11/ BED12 formatted paired-end mapping files]

*Available in [[ZENBU_Howto|Zenbu]] since July 7th, 2011.  Use ''CAGEscan'' as a keyword to find all libraries.
*ZENBU configurations
**https://fantom5-collaboration.gsc.riken.jp/zenbu/gLyphs/#config=mov2PEMUDRuhKoQp1Yl22D;loc=hg19::chr19:50161252..50170707

*BED12 formatted CAGEscan cluster2: https://fantom5-collaboration.gsc.riken.jp/webdav/home/nbertin/CAGEscan-Data-UPDATE_July11/

=== Re-aligned version for the data descriptor ===

* Data was realigned with OP-WORKFLOW-CAGEscan-FANTOM5-v1.0, a derivative of the 2014 pipeline, with hardcoded information to ease processing of the FANTOM5 libraries with Grid Engine. [[https://doi.org/10.6084/m9.figshare.4792666]].  Main reasons for realigning were a) G-correction and b) reproducible research.
* Reads and alignments: [[https://fantom5-collaboration.gsc.riken.jp/files/data/shared/UPDATE_026/f5pipeline/]].

==== Meta CAGEscan assembly ====

The score indicates the number of libraries contributing to the cluster.

<pre>
time cat */CAGEscan_clusters/*.bed |
/home/plessy/src/CAGEscan-Clustering/CAGEscan-Clustering.pl \
  --cluster_file /osc-fs_home/scratch/moirai/plessy/input/hg19.cage_peak_phase1and2combined_anncoord.bed \
  --bin_intersectbed $(which intersectBed) \
  --format bed > CAGEscan_metaclusters.bed
</pre>

<pre>
real	5m27.640s
user	6m23.873s
sys	0m2.293s
</pre>

<pre>
zenbu_upload -url http://fantom.gsc.riken.jp/zenbu -file CAGEscan_metaclusters.bed -assembly hg19 -name "CAGEscan metaclusters" -desc "FANTOM5 CAGEscan metaclusters"
</pre>

==== Zenbu upload of individual libraries ====

Downloaded all the BED12 files from UPDATE_026 in a single directory.

URL decoding, modified from: http://stackoverflow.com/a/10660730/5309786

<pre>
# Returns a string in which the sequences with percent (%) signs followed by
# two hex digits have been replaced with literal characters.
rawurldecode() {

  # This is perhaps a risky gambit, but since all escape characters must be
  # encoded, we can replace %NN with \xNN and pass the lot to printf -b, which
  # will decode hex for us

  printf -v REPLY '%b' "${1//%/\\x}"
  printf "${REPLY}"
}
</pre>

<pre>
for file in *pairs.bed.gz
do
  printf "%s\t%s\t%s\n" \
    $file \
    $(echo $file | grep -o NCig10...) \
    "$(rawurldecode ${file%%.hg19.GCTATA.pairs.bed.gz} | sed 's/\./ /g') upload2016-12-26"
done > CAGEscan_pairs.txt

CAGEscan_clust

for file in *clusters.bed.gz
do
  printf "%s\t%s\t%s\n" \
    $file \
    $(echo $file | grep -o NCig10...) \
    "$(rawurldecode ${file%%.hg19.GCTATA.pairs.bed.gz} | sed 's/\./ /g') upload2016-12-26"
done > CAGEscan_clusters.txt
</pre>

<pre>
zenbu_upload -url http://fantom.gsc.riken.jp/zenbu -assembly hg19 -filelist CAGEscan_pairs.txt -singletag_exp
zenbu_upload -url http://fantom.gsc.riken.jp/zenbu -assembly hg19 -filelist CAGEscan_clusters.txt
</pre>

Available as tracks in view http://fantom.gsc.riken.jp/zenbu/gLyphs/#config=ZkJi4RdBAFhnsudxePrZxD (public collaboration "FANTOM5 CAGEscan").

==== Potential loci of interest ====

<pre>
cat CAGEscan_metaclusters.bed |
  awk '{OFS="\t"} $6 == "+" {print $1, $2, $8, $4, $5, $6}' |
  bedtools subtract -s -a - -b /osc-fs_home/scratch/gmtu/annotation/homo_sapiens/gencode-18/gencode.v18.annotation.genes.bed |
  sort -k5,5n |
  grep p@chr
</pre>

== Pipelines and qsub scripts ==

<pre>
for lib in $(ls /analysisdata/mirrors/F5_file_current/UPDATE_024/f5pipeline/*/*NCig*.5prime.fq.gz | grep -o NCig..... | sort -u )
do
  mkdir -p $lib
  pushd $lib
  sed "s/\[library_id]/$lib/g" ../../CAGEscan_short-reads.xml > $lib.xml
  printf 'MOIRAI_HOME=/home/scratch/moirai/bin\n'     >  $lib.sh
  printf 'DATA_HOME=/home/plessy/pipelines/%s\n' $lib >> $lib.sh
  printf '$MOIRAI_HOME/MoiraiExecute.pl $DATA_HOME/%s.xml\n' $lib >> $lib.sh
  printf '$MOIRAI_HOME/java -classpath $MOIRAI_HOME/Moirai.jar MoiraiUtility log  $DATA_HOME/%s.xml\n' $lib >> $lib.sh
  printf '$MOIRAI_HOME/java -classpath $MOIRAI_HOME/Moirai.jar MoiraiUtility html $DATA_HOME/%s.xml\n' $lib >> $lib.sh
  popd
done
</pre>

[[Category:CAGEscan]]