[biomart-users] Which firewall policies needed in order to query external marts?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[biomart-users] Which firewall policies needed in order to query external marts?

Pablo Ramos
Hi,

I'm trying to make some queries in order to map geneIDs to annotation using biomaRt from within R. While I can successfully connect to biomart using the useEnsembl() function, whenever I query using getBM(...) I get the following error:

Error in value[[3L]](cond) :

 
Request to BioMart web service failed. Verify if you are still connected to the internet.  Alternatively the BioMart web service is temporarily down.

I suspect some firewall policy regarding filters/ports blocked might be the reason, so my main question is which ports should I ask our IT to open in order to successfully get queries using this function.  This problem occurs everytime (I have not been able to get one single answer from the queries, even after trying out in different days), also occurs when using the default useMart() function with www.biomart.org as host and, when I connected to an external internet source not on the faculty I am able to get results for the query, thus my suspicion of a firewall issue. However, I can't just ask them to create a DMZ on my machine.  

I have been trying to track down using netstat the ports R is using while making queries but I only see traffic going out into port 80 to the biomart servers.

sessionInfo() gives me:

> sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.3 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=pt_BR.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=pt_BR.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=pt_BR.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] biomaRt_2.24.0

loaded via a namespace (and not attached):
 [1] IRanges_2.2.7        DBI_0.3.1            parallel_3.2.2      
 [4] RCurl_1.95-4.7       Biobase_2.28.0       AnnotationDbi_1.30.1
 [7] RSQLite_1.0.0        S4Vectors_0.6.3      BiocGenerics_0.14.0 
[10] GenomeInfoDb_1.4.2   stats4_3.2.2         bitops_1.0-6        
[13] XML_3.98-1.3        


Also, I am able to access the servers using wget (as per another post in the group in which connectivity problems where happening):

pablo@bioinfo ~ $ wget -O registry_test.xml http://www.biomart.org/biomart/martservice?type=registry
--2015-08-31 14:22:07--  http://www.biomart.org/biomart/martservice?type=registry
Resolving www.biomart.org (www.biomart.org)... 54.225.80.241
Connecting to www.biomart.org (www.biomart.org)|54.225.80.241|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
Saving to: ‘registry_test.xml’

    [  <=>                                                                                              ] 20.622      90,8KB/s   in 0,2s   

2015-08-31 14:22:11 (90,8 KB/s) - ‘registry_test.xml’ saved [20622]


I am able to open the registry_test.xml file and see the expected output.

The code I am using is:

library("biomaRt")
ensembl <- useEnsembl(biomart="ensembl", dataset="hsapiens_gene_ensembl")
qmyGene <- getBM(attributes = c("description", "uniprot_swissprot"), filters="ensembl_gene_id", values="ENSG00000138821", mart=ensembl)

The useEnsembl() command gives me no errors, only when I try the getBM() function the request error appears.


Thanks!
Pablo



--
You received this message because you are subscribed to the Google Groups "biomart-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Visit this group at http://groups.google.com/group/biomart-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[biomart-users] Re: Which firewall policies needed in order to query external marts?

Ashok Ragavendran
How about if you try list attributes or list filters. It might be a host problem. What happens if you try something like

 ensembl <- useMart(biomart="ENSEMBL_MART_ENSEMBL",host="uswest.ensembl.org",dataset="hsapiens_gene_ensembl") ???
Cheers
  Ashok

On Monday, August 31, 2015 at 2:00:59 PM UTC-4, Pablo Ramos wrote:
Hi,

I'm trying to make some queries in order to map geneIDs to annotation using biomaRt from within R. While I can successfully connect to biomart using the useEnsembl() function, whenever I query using getBM(...) I get the following error:

Error in value[[3L]](cond) :

 
Request to BioMart web service failed. Verify if you are still connected to the internet.  Alternatively the BioMart web service is temporarily down.

I suspect some firewall policy regarding filters/ports blocked might be the reason, so my main question is which ports should I ask our IT to open in order to successfully get queries using this function.  This problem occurs everytime (I have not been able to get one single answer from the queries, even after trying out in different days), also occurs when using the default useMart() function with <a href="http://www.biomart.org" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org as host and, when I connected to an external internet source not on the faculty I am able to get results for the query, thus my suspicion of a firewall issue. However, I can't just ask them to create a DMZ on my machine.  

I have been trying to track down using netstat the ports R is using while making queries but I only see traffic going out into port 80 to the biomart servers.

sessionInfo() gives me:

> sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.3 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=pt_BR.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=pt_BR.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=pt_BR.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] biomaRt_2.24.0

loaded via a namespace (and not attached):
 [1] IRanges_2.2.7        DBI_0.3.1            parallel_3.2.2      
 [4] RCurl_1.95-4.7       Biobase_2.28.0       AnnotationDbi_1.30.1
 [7] RSQLite_1.0.0        S4Vectors_0.6.3      BiocGenerics_0.14.0 
[10] GenomeInfoDb_1.4.2   stats4_3.2.2         bitops_1.0-6        
[13] XML_3.98-1.3        


Also, I am able to access the servers using wget (as per another post in the group in which connectivity problems where happening):

pablo@bioinfo ~ $ wget -O registry_test.xml <a href="http://www.biomart.org/biomart/martservice?type=registry" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org%2Fbiomart%2Fmartservice%3Ftype%3Dregistry\46sa\75D\46sntz\0751\46usg\75AFQjCNH5Tff6c6fTrLYNWObeZJjZ_9GYzw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org%2Fbiomart%2Fmartservice%3Ftype%3Dregistry\46sa\75D\46sntz\0751\46usg\75AFQjCNH5Tff6c6fTrLYNWObeZJjZ_9GYzw&#39;;return true;">http://www.biomart.org/biomart/martservice?type=registry
--2015-08-31 14:22:07--  <a href="http://www.biomart.org/biomart/martservice?type=registry" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org%2Fbiomart%2Fmartservice%3Ftype%3Dregistry\46sa\75D\46sntz\0751\46usg\75AFQjCNH5Tff6c6fTrLYNWObeZJjZ_9GYzw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org%2Fbiomart%2Fmartservice%3Ftype%3Dregistry\46sa\75D\46sntz\0751\46usg\75AFQjCNH5Tff6c6fTrLYNWObeZJjZ_9GYzw&#39;;return true;">http://www.biomart.org/biomart/martservice?type=registry
Resolving <a href="http://www.biomart.org" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org (<a href="http://www.biomart.org" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org)... 54.225.80.241
Connecting to <a href="http://www.biomart.org" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org (<a href="http://www.biomart.org" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org)|54.225.80.241|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
Saving to: ‘registry_test.xml’

    [  <=>                                                                                              ] 20.622      90,8KB/s   in 0,2s   

2015-08-31 14:22:11 (90,8 KB/s) - ‘registry_test.xml’ saved [20622]


I am able to open the registry_test.xml file and see the expected output.

The code I am using is:

library("biomaRt")
ensembl <- useEnsembl(biomart="ensembl", dataset="hsapiens_gene_ensembl")
qmyGene <- getBM(attributes = c("description", "uniprot_swissprot"), filters="ensembl_gene_id", values="ENSG00000138821", mart=ensembl)

The useEnsembl() command gives me no errors, only when I try the getBM() function the request error appears.


Thanks!
Pablo



--
You received this message because you are subscribed to the Google Groups "biomart-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Visit this group at http://groups.google.com/group/biomart-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[biomart-users] Re: Which firewall policies needed in order to query external marts?

Pablo Ramos
Hi Ashok.

I still get the same error when I try using that call.


Error in value[[3L]](cond) : 
  Request to BioMart web service failed. Verify if you are still connected to the internet.  Alternatively the BioMart web service is temporarily down.

I still think it might be a port problem.

Thanks
Pablo

On Wednesday, September 2, 2015 at 10:52:14 AM UTC-3, Ashok Ragavendran wrote:
How about if you try list attributes or list filters. It might be a host problem. What happens if you try something like

 ensembl <- useMart(biomart="ENSEMBL_MART_ENSEMBL",host="<a href="http://uswest.ensembl.org" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fuswest.ensembl.org\46sa\75D\46sntz\0751\46usg\75AFQjCNF28YhtF-816UnivRmTExRR3aTu3g&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fuswest.ensembl.org\46sa\75D\46sntz\0751\46usg\75AFQjCNF28YhtF-816UnivRmTExRR3aTu3g&#39;;return true;">uswest.ensembl.org",dataset="hsapiens_gene_ensembl") ???
Cheers
  Ashok

On Monday, August 31, 2015 at 2:00:59 PM UTC-4, Pablo Ramos wrote:
Hi,

I'm trying to make some queries in order to map geneIDs to annotation using biomaRt from within R. While I can successfully connect to biomart using the useEnsembl() function, whenever I query using getBM(...) I get the following error:

Error in value[[3L]](cond) :

 
Request to BioMart web service failed. Verify if you are still connected to the internet.  Alternatively the BioMart web service is temporarily down.

I suspect some firewall policy regarding filters/ports blocked might be the reason, so my main question is which ports should I ask our IT to open in order to successfully get queries using this function.  This problem occurs everytime (I have not been able to get one single answer from the queries, even after trying out in different days), also occurs when using the default useMart() function with <a href="http://www.biomart.org" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org as host and, when I connected to an external internet source not on the faculty I am able to get results for the query, thus my suspicion of a firewall issue. However, I can't just ask them to create a DMZ on my machine.  

I have been trying to track down using netstat the ports R is using while making queries but I only see traffic going out into port 80 to the biomart servers.

sessionInfo() gives me:

> sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.3 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=pt_BR.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=pt_BR.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=pt_BR.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] biomaRt_2.24.0

loaded via a namespace (and not attached):
 [1] IRanges_2.2.7        DBI_0.3.1            parallel_3.2.2      
 [4] RCurl_1.95-4.7       Biobase_2.28.0       AnnotationDbi_1.30.1
 [7] RSQLite_1.0.0        S4Vectors_0.6.3      BiocGenerics_0.14.0 
[10] GenomeInfoDb_1.4.2   stats4_3.2.2         bitops_1.0-6        
[13] XML_3.98-1.3        


Also, I am able to access the servers using wget (as per another post in the group in which connectivity problems where happening):

pablo@bioinfo ~ $ wget -O registry_test.xml <a href="http://www.biomart.org/biomart/martservice?type=registry" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org%2Fbiomart%2Fmartservice%3Ftype%3Dregistry\46sa\75D\46sntz\0751\46usg\75AFQjCNH5Tff6c6fTrLYNWObeZJjZ_9GYzw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org%2Fbiomart%2Fmartservice%3Ftype%3Dregistry\46sa\75D\46sntz\0751\46usg\75AFQjCNH5Tff6c6fTrLYNWObeZJjZ_9GYzw&#39;;return true;">http://www.biomart.org/biomart/martservice?type=registry
--2015-08-31 14:22:07--  <a href="http://www.biomart.org/biomart/martservice?type=registry" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org%2Fbiomart%2Fmartservice%3Ftype%3Dregistry\46sa\75D\46sntz\0751\46usg\75AFQjCNH5Tff6c6fTrLYNWObeZJjZ_9GYzw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org%2Fbiomart%2Fmartservice%3Ftype%3Dregistry\46sa\75D\46sntz\0751\46usg\75AFQjCNH5Tff6c6fTrLYNWObeZJjZ_9GYzw&#39;;return true;">http://www.biomart.org/biomart/martservice?type=registry
Resolving <a href="http://www.biomart.org" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org (<a href="http://www.biomart.org" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org)... 54.225.80.241
Connecting to <a href="http://www.biomart.org" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org (<a href="http://www.biomart.org" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fwww.biomart.org\46sa\75D\46sntz\0751\46usg\75AFQjCNGTycqpJiNkOBMw43z-5DeV4RRH3w&#39;;return true;">www.biomart.org)|54.225.80.241|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
Saving to: ‘registry_test.xml’

    [  <=>                                                                                              ] 20.622      90,8KB/s   in 0,2s   

2015-08-31 14:22:11 (90,8 KB/s) - ‘registry_test.xml’ saved [20622]


I am able to open the registry_test.xml file and see the expected output.

The code I am using is:

library("biomaRt")
ensembl <- useEnsembl(biomart="ensembl", dataset="hsapiens_gene_ensembl")
qmyGene <- getBM(attributes = c("description", "uniprot_swissprot"), filters="ensembl_gene_id", values="ENSG00000138821", mart=ensembl)

The useEnsembl() command gives me no errors, only when I try the getBM() function the request error appears.


Thanks!
Pablo



--
You received this message because you are subscribed to the Google Groups "biomart-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Visit this group at http://groups.google.com/group/biomart-users.
For more options, visit https://groups.google.com/d/optout.