CAGE Cluster update: Difference between revisions

From Wiki
Jump to navigationJump to search
No edit summary
No edit summary
 
(26 intermediate revisions by 2 users not shown)
Line 8: Line 8:


Note that the set is not finalized yet. The current status is promotome v1.0 RC1, where only the CAGE clusters supported by independent resources (evidences) are included.
Note that the set is not finalized yet. The current status is promotome v1.0 RC1, where only the CAGE clusters supported by independent resources (evidences) are included.

*available data: expression and human readable (but computer-generated) description for each clsuter. See method for the cluster description here: [https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110804-dpi-clusters-expression/hg19-v1.0-RC1/110926-promotome-v1.0-RC1.pdf 110926-promotome-v1.0-RC1.pdf]
⚫
*directory: https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110804-dpi-clusters-expression
*'annotations' discussed by the cluster working group (see below) is not included yet - rough plan is shared and agreed, however, the details are not clearly agreed yet.


=== Genomic coordinates ===
=== Genomic coordinates ===
Line 32: Line 28:
*https://fantom5-collaboration.gsc.riken.jp/webdav/home/Lassmann/TSS_classification/
*https://fantom5-collaboration.gsc.riken.jp/webdav/home/Lassmann/TSS_classification/


== Expression profiles to identify cellular states ==
== Expression profiles to identify cellular states ==


participants: Cesare, Marco, Yishai, Piotor, Kawaji, Christine, Tom, Al
participants: Cesare, Marco, Yishai, Piotr, Kawaji, Christine, Tom, Al


<br>


=== Removal of poor quality libraries ===
=== Removal of poor quality libraries ===
Christine, Tom, Al


see [[Sample_QC]] for the current status


⚫
=== Expression thresholding ===


⚫
=== Cluster thresholding ===
⚫
For a shared set for expression profiles to identify cellular states, we would like to set a threshold of expression (not relying on any other evidences - such as DNase HS site, since it could ignore rare cells). We will decide this until 28th Oct, 2011. The candidates are:


⚫
For a shared set for expression profiles to identify cellular states, we would like to set a threshold of expression (not relying on any other evidences - such as DNase HS site, since it could ignore rare cells). Piotr, Erik vN, Kawaji, and Al explored several parameters, and we concluded as below (Nov 30, 11):
*10 reads at least in a library (while this is arbitrary choise)

*Data driven approach (Piotr will give a try)
* For TSS identification, we use "max counts at CTSS > 2" as a permissive threshold ( [https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110720-dpi-clusters/hg19/thresholding_test/in/ctssMaxCounts/tc.decompose_smoothing_merged_ctssMaxCounts2.bed.gz this threshold on UPDATE_012 DPI clusters] )
* For motif and expression analysis, we use "max counts at CTSS >10 AND max tpm at CTSS > 1" as a robust threshold. ( [https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110720-dpi-clusters/hg19/thresholding_test/in/ctssMaxCounts_ctssTpm/tc.decompose_smoothing_merged_ctssMaxCounts10___tc.decompose_smoothing_merged_ctssMaxTpm1.0.bed.gz this threshold on UPDATE_012 DPI clusters] )

See https://fantom5-collaboration.gsc.riken.jp/wiki/images/b/b4/111131-F5telecon-kawaji.pdf for details.


=== Normalization ===
=== Normalization ===
Line 52: Line 52:
question 1: '''collapsing TSS will change the data structure or not?''' - Yishai and Cesare will compare the normalised data to check whether the thresholding and collapsing the cluster will impact on the normalised structure. Stick with TSS.
question 1: '''collapsing TSS will change the data structure or not?''' - Yishai and Cesare will compare the normalised data to check whether the thresholding and collapsing the cluster will impact on the normalised structure. Stick with TSS.


*results: Yes (at least in Yishai's normalization approach). Yishai compared "Normalization before collapsing" and "Normalization after collapsing", and the latter is better (that is, expression of ACTB and TUBB is relatively uncorrelated).
⚫
question 2: '''Key question: which normalization scheme should be used?''' - Compare power-law method of Piotr's or Yishai with EdgeR TMM RLE and TPM. See the 'Impact of normalization' section below, too.


⚫
<br> question 2: '''Key question: which normalization scheme should be used?''' - Compare power-law method of Piotr's or Yishai with EdgeR TMM RLE and TPM. See the 'Impact of normalization' section below, too.


[Plan and progress]
[Plan and progress]


* step0: set the target data
*step0: set the target data
** DPI cluster, >= 10 reads in a library, at least
**DPI cluster, &gt;= 10 reads in a library, at least
** https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110804-dpi-clusters-expression/hg19-v0.0/tc.chrom_tpm.annotation.max_counts10.txt.gz
**https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110804-dpi-clusters-expression/hg19-v0.0/tc.chrom_tpm.annotation.max_counts10.txt.gz


* step1: different normalization
*step1: normalization results
**Piotr https://fantom5-collaboration.gsc.riken.jp/webdav/home/balwierz/NormalizedKawajiClusterExpression/tc.normalized.piotr.max_counts10.txt.gz
** Piotr
⚫
**TPM https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110804-dpi-clusters-expression/hg19-v0.0/tc.chrom_tpm.annotation.max_counts10.txt.gz
** TPM
**RLE based TPM https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110804-dpi-clusters-expression/hg19-v0.0-edgeR-normalization/tc.max_counts10.rle_tpm.txt.gz
** TMM
**TMM based TPM https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110804-dpi-clusters-expression/hg19-v0.0-edgeR-normalization/tc.max_counts10.tmm_tpm.txt.gz
** RLE
**Yishai's method - Rank-invariant normalization (RINO): https://fantom5-collaboration.gsc.riken.jp/webdav/home/yishai/tc.rino.chrom_tpm.annotation.txt.gz


* Step2: evaluation
*Step2: Systematic evaluation - Cesare, Marco, Yishai
**Box plot - https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/111117-normalization-evaluation/boxplot/boxplot.4methods.png [https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/111117-normalization-evaluation/Yishai-boxplot.3methods.readme.txt (readme)]
** Systematic evaluation - Cesare, Marco, Yishai
**scatter plot and R^2 on a time course (Marco) - https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/111117-normalization-evaluation/Marco-timecourse-scatter-R2/ [https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/111117-normalization-evaluation/Marco-timecourse-scatter-R2.readme.txt (readme)]
** Biological evaluation - Christine, Tom, Al
**MA plots on 6 random pairs https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/111117-normalization-evaluation/Marco-MA/

*Step3: Inspection from biological perspective - Christine, Tom, Al
** Pearson's correlation-based network https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/111117-normalization-evaluation/Norm_90P_summary_file.xlsx


=== Impact of Normalization on FANTOM5 hCAGE promoterome: Implementation Plan (0.1) ===
=== Impact of Normalization on FANTOM5 hCAGE promoterome: Implementation Plan (0.1) ===

#We (Trento, Columbia – Piotr is invited) will start implementing the normalization check scheme. It means building a generic script to run on DPI TSS tag cluster set(s) when they are ready. Can be run on any other clustering scheme for TSS, or before the clustering. Hopefully
#We (Trento, Columbia – Piotr is invited) will start implementing the normalization check scheme. It means building a generic script to run on DPI TSS tag cluster set(s) when they are ready. Can be run on any other clustering scheme for TSS, or before the clustering. Hopefully
##Run power-law fits on single libraries, compare with pooled data; check range of parameters, infer stability on pooled; compare with QC indicators on single libraries; consider; check for batch effect. Start from YB script; Piotr’s version as available.
##Run power-law fits on single libraries, compare with pooled data; check range of parameters, infer stability on pooled; compare with QC indicators on single libraries; consider; check for batch effect. Start from YB script; Piotr’s version as available.
Line 79: Line 86:
#Apply on DPI tag cluster set(s) as they are ready and report, to be used for main paper
#Apply on DPI tag cluster set(s) as they are ready and report, to be used for main paper
#It can make sense to reuse all material for the “peak clustering and normalization” satellite paper to submit aside the main paper, or later
#It can make sense to reuse all material for the “peak clustering and normalization” satellite paper to submit aside the main paper, or later

=== Description of clusters ===
* assigned description (with association to gene models) according to [https://fantom5-collaboration.gsc.riken.jp/wiki/index.php/File:111208-telecon-kawaji.pdf this process]
** [https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110720-dpi-clusters/hg19/agreed_thresholds/ data directory]
** [https://fantom5-collaboration.gsc.riken.jp/ucsc/cgi-bin/hgTracks?db=hg19&org=human&hgt.customText=https://oscf5:0114collab@fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110720-dpi-clusters/hg19/ucsc_config.txt config on UCSC mirror]


== Cluster annotations discussed in [[Tag Cluster Annotation|the cluster working group]] ==
== Cluster annotations discussed in [[Tag Cluster Annotation|the cluster working group]] ==

Latest revision as of 18:06, 15 December 2011

This page is to update information about CAGE clusters on UPDATE_012

CAGE clusters for the main paper

Please contact to fantom5-wp4@gsc.riken.jp about the whole status, and corresponding providers for individual questions.

'Promotome' set

Note that the set is not finalized yet. The current status is promotome v1.0 RC1, where only the CAGE clusters supported by independent resources (evidences) are included.

Genomic coordinates

https://fantom5-collaboration.gsc.riken.jp/webdav/home/kawaji/110720-dpi-clusters/

ZENBU configuration

human and mouse

TSS classifier

[Sebastian et al.]

[Timo]

Expression profiles to identify cellular states

participants: Cesare, Marco, Yishai, Piotr, Kawaji, Christine, Tom, Al


Removal of poor quality libraries

see Sample_QC for the current status


Cluster thresholding

For a shared set for expression profiles to identify cellular states, we would like to set a threshold of expression (not relying on any other evidences - such as DNase HS site, since it could ignore rare cells). Piotr, Erik vN, Kawaji, and Al explored several parameters, and we concluded as below (Nov 30, 11):

See https://fantom5-collaboration.gsc.riken.jp/wiki/images/b/b4/111131-F5telecon-kawaji.pdf for details.

Normalization

question 1: collapsing TSS will change the data structure or not? - Yishai and Cesare will compare the normalised data to check whether the thresholding and collapsing the cluster will impact on the normalised structure. Stick with TSS.

  • results: Yes (at least in Yishai's normalization approach). Yishai compared "Normalization before collapsing" and "Normalization after collapsing", and the latter is better (that is, expression of ACTB and TUBB is relatively uncorrelated).


question 2: Key question: which normalization scheme should be used? - Compare power-law method of Piotr's or Yishai with EdgeR TMM RLE and TPM. See the 'Impact of normalization' section below, too.

[Plan and progress]

Impact of Normalization on FANTOM5 hCAGE promoterome: Implementation Plan (0.1)

  1. We (Trento, Columbia – Piotr is invited) will start implementing the normalization check scheme. It means building a generic script to run on DPI TSS tag cluster set(s) when they are ready. Can be run on any other clustering scheme for TSS, or before the clustering. Hopefully
    1. Run power-law fits on single libraries, compare with pooled data; check range of parameters, infer stability on pooled; compare with QC indicators on single libraries; consider; check for batch effect. Start from YB script; Piotr’s version as available.
    2. Match with normalization and QC methods from edgeR, also in script form. Keep it generic, automate.
    3. Apply on existing tag DPI clusters. Run also before and after other clustering. Prepare reporting scheme.
    4. Verify impact on a few examples of downstream analysis, e.g. quality/stability/accuracy of classifiers and networks
  2. Apply on DPI tag cluster set(s) as they are ready and report, to be used for main paper
  3. It can make sense to reuse all material for the “peak clustering and normalization” satellite paper to submit aside the main paper, or later

Description of clusters

Cluster annotations discussed in the cluster working group

Method Description

Transcript model derived annotation protocol

Annotation Results

Human Decomposition-based Peak Identification (DPI) cluster
  • https://fantom5-collaboration.gsc.riken.jp/webdav/home/nbertin/CAGE-Tag-Cluster-Annotation_Aug11/tc.decompose_smoothing_merged.hg19.annotations/
    • tc.decompose_smoothing_merged.hg19.CpGislands.annotated.osc.gz
    • tc.decompose_smoothing_merged.hg19.EST.annotated.osc.gz
    • tc.decompose_smoothing_merged.hg19.Ensembl.non_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.hg19.Ensembl.protein_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.hg19.F5_human_lncRNAome.annotated.osc.gz
    • tc.decompose_smoothing_merged.hg19.RefSeq.non_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.hg19.RefSeq.protein_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.hg19.TBP_JASPAR_CORE_MA0108.2.annotated.osc.gz
    • tc.decompose_smoothing_merged.hg19.gencode-pseudo.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.hg19.gencode.non_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.hg19.gencode.protein_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.hg19.knownGene.non_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.hg19.knownGene.protein_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.hg19.mRNA.annotated.osc.gz
    • tc.decompose_smoothing_merged.hg19.rmsk.annotated.repClass.repFamily.osc.gz
Mouse Decomposition-based Peak Identification (DPI) cluster
  • https://fantom5-collaboration.gsc.riken.jp/webdav/home/nbertin/CAGE-Tag-Cluster-Annotation_Aug11/tc.decompose_smoothing_merged.mm9.annotations/
    • tc.decompose_smoothing_merged.mm9.CpGislands.annotated.osc.gz
    • tc.decompose_smoothing_merged.mm9.EST.annotated.osc.gz
    • tc.decompose_smoothing_merged.mm9.Ensembl.non_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.mm9.Ensembl.protein_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.mm9.RefSeq.non_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.mm9.RefSeq.protein_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.mm9.TBP_JASPAR_CORE_MA0108.2.annotated.osc.gz
    • tc.decompose_smoothing_merged.mm9.knownGene.non_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.mm9.knownGene.protein_coding.annotated.sym.osc.gz
    • tc.decompose_smoothing_merged.mm9.mRNA.annotated.osc.gz
    • tc.decompose_smoothing_merged.mm9.rmsk.annotated.repClass.repFamily.osc.gz


Other clusters on UPDATE_012

Please don't hesitate to use/produce other clusters for other purposes

Related pages

Retrieved from "http://fantom5-collaboration.gsc.riken.jp/wiki/index.php?title=CAGE_Cluster_update&oldid=4186"