can not access the GDC dataset

[JianGuoZhou3]

> XenaGenerate(subset = XenaHostNames=="gdcHub") %>% 
+   XenaFilter(filterDatasets = "methylation|phenotype") %>% 
+   XenaFilter(filterDatasets = "UCS") -> df_todo
> XenaQuery(df_todo) %>%
+   XenaDownload() -> xe_download
This will check url status, please be patient.
All downloaded files will under directory /tmp/RtmpciLecI.
The 'trans_slash' option is FALSE, keep same directory structure as Xena.
Creating directories for datasets...
'/tmp/RtmpciLecI/TCGA-UCS/Xena_Matrices' already exists'/tmp/RtmpciLecI/TCGA-UCS/Xena_Matrices' already exists/tmp/RtmpciLecI/TCGA-UCS/Xena_Matrices/TCGA-UCS.GDC_phenotype.tsv.gz, the file has been download!
/tmp/RtmpciLecI/TCGA-UCS/Xena_Matrices/TCGA-UCS.methylation450.tsv.gz, the file has been download!

[JianGuoZhou3]

anthor question is the package can continue download or not ?

> XenaQuery(df_todo) %>%
+   XenaDownload() -> xe_download
This will check url status, please be patient.
All downloaded files will under directory /tmp/RtmpciLecI.
The 'trans_slash' option is FALSE, keep same directory structure as Xena.
Creating directories for datasets...
'/tmp/RtmpciLecI/TCGA.HNSC.sampleMap' already existsDownloading TCGA.HNSC.sampleMap/HumanMethylation450.gz
trying URL 'https://tcga.xenahubs.net/download/TCGA.HNSC.sampleMap/HumanMethylation450.gz'
Content type 'application/gzip' length 516443612 bytes (492.5 MB)
=======================================
downloaded 388.7 MB

downloaded length 407552000 != reported length 516443612URL 'https://tcga.xenahubs.net/download/TCGA.HNSC.sampleMap/HumanMethylation450.gz': status was 'Transferred a partial file'Can not find fileTCGA.HNSC.sampleMap/HumanMethylation450.gz, this file maybe not compressed.
Try downloading fileTCGA.HNSC.sampleMap/HumanMethylation450...
trying URL 'https://tcga.xenahubs.net/download/TCGA.HNSC.sampleMap/HumanMethylation450'
cannot open URL 'https://tcga.xenahubs.net/download/TCGA.HNSC.sampleMap/HumanMethylation450': HTTP status was '404 Not Found'Error in download.file(x[3], destfile = x[5], ...) : 
  cannot open URL 'https://tcga.xenahubs.net/download/TCGA.HNSC.sampleMap/HumanMethylation450'

[ShixiangWang]

@jianguozhouzunyimedicaluniversity Thank you for reporting this issue. I can reproduce it.

It seems GDC hub has a different parse rule. I will try to fix this.

https://gdc.xenahubs.net/download/TCGA-UCS.GDC_phenotype.tsv.gz  # Actual link

https://gdc.xenahubs.net/download/TCGA-UCS/Xena_Matrices/TCGA-UCS.GDC_phenotype.tsv.gz  # The link this package try to download

By the way, download process cannot continue. I will take a look if it can be implemented.

Thanks again.
Shixiang

[JianGuoZhou3]

fine. shixiang, you can check the dong's package (GDCRNATools).This package can continue download.best wishes Jian-Guo ZHOU Department of Oncology, Affiliated Hospital of Zunyi Medical University Address: NO.149, Dalian Road, Zunyi City, Guizhou Province, China, 563000 Email: [email protected] Shixiang Wang <[email protected]> 于 2019年9月26日星期四 GMT+2 下午05:58:50写道： Closed #12 via 5daf250. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[ShixiangWang]

你可以从 github 安装最新版本看下问题解决了吗

[JianGuoZhou3]

OK. 我下班回家就看。到时候邮件给你回复。thanks Jian-Guo ZHOU Department of Oncology, Affiliated Hospital of Zunyi Medical University Address: NO.149, Dalian Road, Zunyi City, Guizhou Province, China, 563000 Email: [email protected] Shixiang Wang <[email protected]> 于 2019年9月26日星期四 GMT+2 下午06:03:37写道： 你可以从 github 安装最新版本看下问题解决了吗 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[ShixiangWang]

我测试过了，目前服务器端不支持断点续传。所以即使我写好了代码也成功不了。我已经写邮件问 UCSC Xena 了，有消息回复你。

[JianGuoZhou3]

没事。Jian-Guo ZHOU M.D.Department of Oncology, Affiliated Hospital of Zunyi Medical UniversityAddress: NO.149, Dalian Road, Zunyi City, Guizhou Province, China, 563000Email: [email protected] 原始邮件 --------发件人： Shixiang Wang <[email protected]>日期： 2019年9月26日周四 19:20收件人： ropensci/UCSCXenaTools <[email protected]>抄送： Jian-Guo Zhou <[email protected]>, Mention <[email protected]>主 题： Re: [ropensci/UCSCXenaTools] can not access the GDC dataset (#12)我测试过了，目前服务器端不支持断点续传。所以即使我写好了代码也成功不了。我已经写邮件问 UCSC Xena 了，有消息回复你。 —You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

[ShixiangWang]

@jianguozhouzunyimedicaluniversity UCSCXena 已经支持断点续传。

下面是一个示例代码

library(UCSCXenaTools)
xe = XenaGenerate(subset = XenaDatasets == "TcgaTargetGtex_expected_count")
xe
xq = XenaQuery(xe)
# You cannot resume from breakpoint in default mode
XenaDownload(xq, destdir = "~/test/", force = TRUE)
# You can do it with 'curl' command
XenaDownload(xq, destdir = "~/test/", method = "curl", extra = "-C -", force = TRUE)
# You can do it with 'wget' command
XenaDownload(xq, destdir = "~/test/", method = "wget", extra = "-c", force = TRUE)

[JianGuoZhou3]

cool, thanks Shixiang. Jian-Guo ZHOU Shixiang Wang <[email protected]> 于 2019年10月8日星期二 GMT+2 上午11:49:38写道： @jianguozhouzunyimedicaluniversity UCSCXena 已经支持断点续传。 下面是一个示例代码 library(UCSCXenaTools) xe = XenaGenerate(subset = XenaDatasets == "TcgaTargetGtex_expected_count") xe xq = XenaQuery(xe) # You cannot resume from breakpoint in default mode XenaDownload(xq, destdir = "~/test/", force = TRUE) # You can do it with 'curl' command XenaDownload(xq, destdir = "~/test/", method = "curl", extra = "-C -", force = TRUE) # You can do it with 'wget' command XenaDownload(xq, destdir = "~/test/", method = "wget", extra = "-c", force = TRUE) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.