[biomart-users] "file descriptor" problem on RHEL 7

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[biomart-users] "file descriptor" problem on RHEL 7

zhilianghu
I have set up the most recent Biomart on a RedHat 7 system with
  > git clone https://github.com/biomart/BioMart.git --branch 0.9.0

compiled with 'ant' and started as in
  > cd dist
  > ./scripts/biomart-server.sh

Immediately it generates tons of open files (before any data is set up):

  > lsof | grep java | wc -l
15745
  > lsof | grep biomart | wc -l
282

... I was telling the story in a reverse way - the problem initially was
there were often system complains that a process could not write to "/tmp"
from time to time.  A "lsof | wc -l" shows it's over 19,000. I didn't
want to blindly raise the limit. That led me to dig deeper to find not
that it was caused by not enough file descriptors left on the system. When
stop Biomart it went down to ~3,000, comparable to our other servers.

This is consistantly repeatable.

Biomart runs well on RedHat 6 on our old server.  Anything special for
RH7 that makes Biomart loops out too many open files?

Zhiliang

--
You received this message because you are subscribed to the Google Groups "biomart-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Visit this group at https://groups.google.com/group/biomart-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [biomart-users] "file descriptor" problem on RHEL 7

Arek Kasprzyk
Hi Zhiliang,

I am not aware of anything peculiar with regards to RH7 but i have not personally tested there

a.


On 29 July 2016 at 21:04, Zhiliang Hu <[hidden email]> wrote:
I have set up the most recent Biomart on a RedHat 7 system with
 > git clone https://github.com/biomart/BioMart.git --branch 0.9.0

compiled with 'ant' and started as in
 > cd dist
 > ./scripts/biomart-server.sh

Immediately it generates tons of open files (before any data is set up):

 > lsof | grep java | wc -l
15745
 > lsof | grep biomart | wc -l
282

... I was telling the story in a reverse way - the problem initially was
there were often system complains that a process could not write to "/tmp"
from time to time.  A "lsof | wc -l" shows it's over 19,000. I didn't
want to blindly raise the limit. That led me to dig deeper to find not
that it was caused by not enough file descriptors left on the system. When
stop Biomart it went down to ~3,000, comparable to our other servers.

This is consistantly repeatable.

Biomart runs well on RedHat 6 on our old server.  Anything special for
RH7 that makes Biomart loops out too many open files?

Zhiliang

--
You received this message because you are subscribed to the Google Groups "biomart-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Visit this group at https://groups.google.com/group/biomart-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "biomart-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Visit this group at https://groups.google.com/group/biomart-users.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [biomart-users] "file descriptor" problem on RHEL 7

zhilianghu
In reply to this post by zhilianghu
Arek,

Could it be a java problem (on RH7)?  If you Google "java too many open
files on centos 7" you will get a number of cases, such this one:
https://www.jayway.com/2012/02/11/how-to-really-fix-the-too-many-open-files-problem-for-tomcat-in-ubuntu/)
and this one: http://www.adarshr.com/too-many-open-files

I am not a java person thus can't say much. Any others on the list who
encounter similar problem?

Zhiliang

On Wed, 10 Aug 2016, Arek Kasprzyk wrote:

> Date: Wed, 10 Aug 2016 09:58:48 +0100
> From: Arek Kasprzyk <[hidden email]>
> To: Zhiliang Hu <[hidden email]>
> Cc: "[hidden email]" <[hidden email]>
> Subject: Re: [biomart-users] "file descriptor" problem on RHEL 7
>
> --001a1146ec9c94ba8f0539b3db6d
> Content-Type: text/plain; charset=UTF-8
>
> Hi Zhiliang,
>
> I am not aware of anything peculiar with regards to RH7 but i have not
> personally tested there
>
> a.
>
>
> On 29 July 2016 at 21:04, Zhiliang Hu <[hidden email]> wrote:
>
>> I have set up the most recent Biomart on a RedHat 7 system with
>> > git clone https://github.com/biomart/BioMart.git --branch 0.9.0
>>
>> compiled with 'ant' and started as in
>> > cd dist
>> > ./scripts/biomart-server.sh
>>
>> Immediately it generates tons of open files (before any data is set up):
>>
>> > lsof | grep java | wc -l
>> 15745
>> > lsof | grep biomart | wc -l
>> 282
>>
>> ... I was telling the story in a reverse way - the problem initially was
>> there were often system complains that a process could not write to "/tmp"
>> from time to time.  A "lsof | wc -l" shows it's over 19,000. I didn't
>> want to blindly raise the limit. That led me to dig deeper to find not
>> that it was caused by not enough file descriptors left on the system. When
>> stop Biomart it went down to ~3,000, comparable to our other servers.
>>
>> This is consistantly repeatable.
>>
>> Biomart runs well on RedHat 6 on our old server.  Anything special for
>> RH7 that makes Biomart loops out too many open files?
>>
>> Zhiliang
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "biomart-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [hidden email].
>> Visit this group at https://groups.google.com/group/biomart-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

--
You received this message because you are subscribed to the Google Groups "biomart-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Visit this group at https://groups.google.com/group/biomart-users.
For more options, visit https://groups.google.com/d/optout.