[BioMart Users] Problems with setting up ICGC mart

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[BioMart Users] Problems with setting up ICGC mart

Rebecca Ewing
Hi

I have our version of the ICGC mart up and running. However, it is taking over 40 minutes for the server to start. Is there anything I can do to speed it up?

Thanks

Rebecca

--
 The Wellcome Trust Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.
_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problems with setting up ICGC mart

Brett Whitty
Hi Rebecca,

That does seem unusual. Can you please tell me what version of BioMart
you're running?
Also, what are the specs if the server you're running on? You should have
a look at the memory usage of the biomart java process as it's starting up
to see if there's a problem with memory usage and available memory on the
server.

Also, it might be worth checking the log files under 'dist/logs' for any
informative messages on startup; maybe post anything unusual you might
find to the list. You can also try increasing the logging level to 'DEBUG'
by adding 'log.level = DEBUG' in your biomart.properties file and see if
that reveals anything more.

Regards,

Brett

On 5/5/11 10:05 AM, "Rebecca Shepherd" <[hidden email]> wrote:

>I have our version of the ICGC mart up and running. However, it is taking
>over 40 minutes for the server to start. Is there anything I can do to
>speed it up?

_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problems with setting up ICGC mart

Rebecca Ewing
Hi

I am using Biomart 0.8 rc5. I have sorted out the problem with the server but am unable to get any data. I am getting the following error in the log file:

java.net.ConnectException: Connection timed out
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
        at java.net.Socket.connect(Socket.java:519)
        at java.net.Socket.connect(Socket.java:469)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
        at sun.net.www.http.HttpClient.New(HttpClient.java:306)
        at sun.net.www.http.HttpClient.New(HttpClient.java:323)
        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:852)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:793)
        at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:718)
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:896)
        at org.biomart.queryEngine.SubQuery.executeMartServiceQuery(SubQuery.java:773)
        at org.biomart.queryEngine.SubQuery.executeQuery(SubQuery.java:698)
        at org.biomart.queryEngine.QueryRunnerThread.recurseSubQueries(QueryRunnerThread.java:173)
        at org.biomart.queryEngine.QueryRunnerThread.call(QueryRunnerThread.java:75)
        at org.biomart.queryEngine.QueryRunnerThread.call(QueryRunnerThread.java:36)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
2011-05-06 15:47:28,880 WARN  [1763281041@qtp-1697022030-7:Slf4jLog.java:76]: /martservice/results: java.io.IOException: Closed

I am having a similar problem with our COSMIC mart. It seems to work fine using rc4 but I don't get any data using rc5.

Do you have any idea what might be wrong?

Thanks

Rebecca

On 5 May 2011, at 15:32, Brett Whitty wrote:

> Hi Rebecca,
>
> That does seem unusual. Can you please tell me what version of BioMart
> you're running?
> Also, what are the specs if the server you're running on? You should have
> a look at the memory usage of the biomart java process as it's starting up
> to see if there's a problem with memory usage and available memory on the
> server.
>
> Also, it might be worth checking the log files under 'dist/logs' for any
> informative messages on startup; maybe post anything unusual you might
> find to the list. You can also try increasing the logging level to 'DEBUG'
> by adding 'log.level = DEBUG' in your biomart.properties file and see if
> that reveals anything more.
>
> Regards,
>
> Brett
>
> On 5/5/11 10:05 AM, "Rebecca Shepherd" <[hidden email]> wrote:
>
>> I have our version of the ICGC mart up and running. However, it is taking
>> over 40 minutes for the server to start. Is there anything I can do to
>> speed it up?
>



--
 The Wellcome Trust Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.
_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problems with setting up ICGC mart

Brett Whitty
Hi Rebecca,

Is it possible there may be some network configuration issues with your
server preventing it from accessing URL-based data sources?

Check the log files for a message that contains the URL that the
queryEngine was trying to access (should contain the text string 'URL' ---
you may have to set log level to DEBUG). See if you can access the URL
that the query engine is trying to access manually (via your browser, or
something like curl or wget on the server itself). If that works fine,
then at least we know the remote server is not the problem.

Let me know what you find, and if you'd like to include the DEBUG level
log snippet that includes the failed query URL as well that might be
helpful.

Regards,

Brett

On 5/6/11 10:53 AM, "Rebecca Shepherd" <[hidden email]> wrote:

>Hi
>
>I am using Biomart 0.8 rc5. I have sorted out the problem with the server
>but am unable to get any data. I am getting the following error in the
>log file:
>
>java.net.ConnectException: Connection timed out
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
> at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:519)
> at java.net.Socket.connect(Socket.java:469)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
> at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
> at sun.net.www.http.HttpClient.New(HttpClient.java:306)
> at sun.net.www.http.HttpClient.New(HttpClient.java:323)
> at
>sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnec
>tion.java:852)
> at
>sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection
>.java:793)
> at
>sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java
>:718)
> at
>sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnect
>ion.java:896)
> at
>org.biomart.queryEngine.SubQuery.executeMartServiceQuery(SubQuery.java:773
>)
> at org.biomart.queryEngine.SubQuery.executeQuery(SubQuery.java:698)
> at
>org.biomart.queryEngine.QueryRunnerThread.recurseSubQueries(QueryRunnerThr
>ead.java:173)
> at
>org.biomart.queryEngine.QueryRunnerThread.call(QueryRunnerThread.java:75)
> at
>org.biomart.queryEngine.QueryRunnerThread.call(QueryRunnerThread.java:36)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.
>java:886)
> at
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:908)
> at java.lang.Thread.run(Thread.java:619)
>2011-05-06 15:47:28,880 WARN
>[1763281041@qtp-1697022030-7:Slf4jLog.java:76]: /martservice/results:
>java.io.IOException: Closed
>
>I am having a similar problem with our COSMIC mart. It seems to work fine
>using rc4 but I don't get any data using rc5.
>
>Do you have any idea what might be wrong?
>
>Thanks
>
>Rebecca
>
>On 5 May 2011, at 15:32, Brett Whitty wrote:
>
>> Hi Rebecca,
>>
>> That does seem unusual. Can you please tell me what version of BioMart
>> you're running?
>> Also, what are the specs if the server you're running on? You should
>>have
>> a look at the memory usage of the biomart java process as it's starting
>>up
>> to see if there's a problem with memory usage and available memory on
>>the
>> server.
>>
>> Also, it might be worth checking the log files under 'dist/logs' for any
>> informative messages on startup; maybe post anything unusual you might
>> find to the list. You can also try increasing the logging level to
>>'DEBUG'
>> by adding 'log.level = DEBUG' in your biomart.properties file and see if
>> that reveals anything more.
>>
>> Regards,
>>
>> Brett
>>
>> On 5/5/11 10:05 AM, "Rebecca Shepherd" <[hidden email]> wrote:
>>
>>> I have our version of the ICGC mart up and running. However, it is
>>>taking
>>> over 40 minutes for the server to start. Is there anything I can do to
>>> speed it up?
>>
>
>
>
>--
> The Wellcome Trust Sanger Institute is operated by Genome Research
> Limited, a charity registered in England with number 1021457 and a
> company registered in England with number 2742969, whose registered
> office is 215 Euston Road, London, NW1 2BE.

_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problems with setting up ICGC mart

Brett Whitty
In reply to this post by Rebecca Ewing
Hi Rebecca,

Depending on the server configuration both DB sources and HTTP could be
affected by network issues.

The easiest way to figure this out might be if you do the following:
- shut down your server
- remove the current log files
- add 'log.level = DEBUG' to your biomart.properties file (if it's not
already there)
- start your server back up
- access the server to ensure the server log will contain some of the
failing queries

If you can then send me the contents of that log file directly to my email
address as a file attachment it may help to try to solve the problem
having that information.

Brett

On 5/6/11 11:23 AM, "Rebecca Shepherd" <[hidden email]> wrote:

>Hi Brett
>
>Our COSMIC mart data comes from our database and I can't get any data
>back for that with rc5, but I can with rc4.
>
>Thanks
>
>Rebecca
>
>On 6 May 2011, at 16:21, Brett Whitty wrote:
>
>> Hi Rebecca,
>>
>> Is it possible there may be some network configuration issues with your
>> server preventing it from accessing URL-based data sources?
>>
>> Check the log files for a message that contains the URL that the
>> queryEngine was trying to access (should contain the text string 'URL'
>>---
>> you may have to set log level to DEBUG). See if you can access the URL
>> that the query engine is trying to access manually (via your browser, or
>> something like curl or wget on the server itself). If that works fine,
>> then at least we know the remote server is not the problem.
>>
>> Let me know what you find, and if you'd like to include the DEBUG level
>> log snippet that includes the failed query URL as well that might be
>> helpful.
>>
>> Regards,
>>
>> Brett
>>
>> On 5/6/11 10:53 AM, "Rebecca Shepherd" <[hidden email]> wrote:
>>
>>> Hi
>>>
>>> I am using Biomart 0.8 rc5. I have sorted out the problem with the
>>>server
>>> but am unable to get any data. I am getting the following error in the
>>> log file:
>>>
>>> java.net.ConnectException: Connection timed out
>>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>>> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
>>> at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
>>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>>> at java.net.Socket.connect(Socket.java:519)
>>> at java.net.Socket.connect(Socket.java:469)
>>> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
>>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
>>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
>>> at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
>>> at sun.net.www.http.HttpClient.New(HttpClient.java:306)
>>> at sun.net.www.http.HttpClient.New(HttpClient.java:323)
>>> at
>>>
>>>sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConn
>>>ec
>>> tion.java:852)
>>> at
>>>
>>>sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnecti
>>>on
>>> .java:793)
>>> at
>>>
>>>sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.ja
>>>va
>>> :718)
>>> at
>>>
>>>sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConne
>>>ct
>>> ion.java:896)
>>> at
>>>
>>>org.biomart.queryEngine.SubQuery.executeMartServiceQuery(SubQuery.java:7
>>>73
>>> )
>>> at org.biomart.queryEngine.SubQuery.executeQuery(SubQuery.java:698)
>>> at
>>>
>>>org.biomart.queryEngine.QueryRunnerThread.recurseSubQueries(QueryRunnerT
>>>hr
>>> ead.java:173)
>>> at
>>>
>>>org.biomart.queryEngine.QueryRunnerThread.call(QueryRunnerThread.java:75
>>>)
>>> at
>>>
>>>org.biomart.queryEngine.QueryRunnerThread.call(QueryRunnerThread.java:36
>>>)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at
>>>
>>>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
>>>r.
>>> java:886)
>>> at
>>>
>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
>>>va
>>> :908)
>>> at java.lang.Thread.run(Thread.java:619)
>>> 2011-05-06 15:47:28,880 WARN
>>> [1763281041@qtp-1697022030-7:Slf4jLog.java:76]: /martservice/results:
>>> java.io.IOException: Closed
>>>
>>> I am having a similar problem with our COSMIC mart. It seems to work
>>>fine
>>> using rc4 but I don't get any data using rc5.
>>>
>>> Do you have any idea what might be wrong?
>>>
>>> Thanks
>>>
>>> Rebecca
>>>
>>> On 5 May 2011, at 15:32, Brett Whitty wrote:
>>>
>>>> Hi Rebecca,
>>>>
>>>> That does seem unusual. Can you please tell me what version of BioMart
>>>> you're running?
>>>> Also, what are the specs if the server you're running on? You should
>>>> have
>>>> a look at the memory usage of the biomart java process as it's
>>>>starting
>>>> up
>>>> to see if there's a problem with memory usage and available memory on
>>>> the
>>>> server.
>>>>
>>>> Also, it might be worth checking the log files under 'dist/logs' for
>>>>any
>>>> informative messages on startup; maybe post anything unusual you might
>>>> find to the list. You can also try increasing the logging level to
>>>> 'DEBUG'
>>>> by adding 'log.level = DEBUG' in your biomart.properties file and see
>>>>if
>>>> that reveals anything more.
>>>>
>>>> Regards,
>>>>
>>>> Brett
>>>>
>>>> On 5/5/11 10:05 AM, "Rebecca Shepherd" <[hidden email]> wrote:
>>>>
>>>>> I have our version of the ICGC mart up and running. However, it is
>>>>> taking
>>>>> over 40 minutes for the server to start. Is there anything I can do
>>>>>to
>>>>> speed it up?
>>>>
>>>
>>>
>>>
>>> --
>>> The Wellcome Trust Sanger Institute is operated by Genome Research
>>> Limited, a charity registered in England with number 1021457 and a
>>> company registered in England with number 2742969, whose registered
>>> office is 215 Euston Road, London, NW1 2BE.
>>
>
>
>
>--
> The Wellcome Trust Sanger Institute is operated by Genome Research
> Limited, a charity registered in England with number 1021457 and a
> company registered in England with number 2742969, whose registered
> office is 215 Euston Road, London, NW1 2BE.

_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users