Bowtie2 index building

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Bowtie2 index building

Deepak Tanwar
Hello,

I am trying to build Bowtie2 index for hg19.
I a using Docker image of Galaxy.

Computer I am using have 12 cores and 64 GB of RAM.

Error I am getting is:

Fatal error: Exit code 1 ()
Building a SMALL index
Could not allocate space for a joined string of 2897310462 elements.
Switching to a packed string representation.
Building a SMALL index
Could not find approrpiate bmax/dcv settings for building this index.
Already tried a packed string representation.
Please try indexing this reference on a computer with more memory.
Error: Encountered internal Bowtie 2 exception (#1)
Command: bowtie2-build --wrapper basic-0 /export/galaxy-central/database/job_working_directory/000/113/dataset_80_files/hg19 fa.fa hg19 fa 
Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
Error building index. 

Please help!

Thank you,
Deepak


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bowtie2 index building

Björn Grüning-3
Hi Deepak,

how are you running this Container?
Can you check how many memory you have inside the container?

Cheers,
Bjoern

On 11.08.2017 15:35, Deepak Tanwar wrote:

> Hello,
>
> I am trying to build Bowtie2 index for hg19.
> I a using Docker image of Galaxy.
>
> Computer I am using have 12 cores and 64 GB of RAM.
>
> Error I am getting is:
>
> Fatal error: Exit code 1 ()
> Building a SMALL index
> Could not allocate space for a joined string of 2897310462 elements.
> Switching to a packed string representation.
> Building a SMALL index
> Could not find approrpiate bmax/dcv settings for building this index.
> Already tried a packed string representation.
> Please try indexing this reference on a computer with more memory.
> Error: Encountered internal Bowtie 2 exception (#1)
> Command: bowtie2-build --wrapper basic-0
> /export/galaxy-central/database/job_working_directory/000/113/dataset_80_files/hg19
> fa.fa hg19 fa
> Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
> Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
> Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
> Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
> Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
> Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
> Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
> Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
> Error building index.
>
> Please help!
>
> Thank you,
> Deepak
>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bowtie2 index building

Deepak Tanwar
Hi Björn,

If I do docker stats:
CONTAINER           CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
dd4da10c0a54        6.16%               690.9MiB / 995.8MiB   69.38%              158kB / 785kB       11.4GB / 81MB       106

And, after ssh into docker machine:

Filesystem                Size      Used Available Use% Mounted on
tmpfs                   896.2M    186.6M    709.6M  21% /
tmpfs                   497.9M    476.0K    497.4M   0% /dev/shm
/dev/sda1               282.7G     14.9G    253.2G   6% /mnt/sda1
cgroup                  497.9M         0    497.9M   0% /sys/fs/cgroup
Users                   931.0G    683.8G    247.2G  73% /Users
/dev/sda1               282.7G     14.9G    253.2G   6% /mnt/sda1/var/lib/docker/aufs

Best regards,
Deepak


On Aug 11, 2017, at 9:39 AM, Björn Grüning <[hidden email]> wrote:

Hi Deepak,

how are you running this Container?
Can you check how many memory you have inside the container?

Cheers,
Bjoern

On 11.08.2017 15:35, Deepak Tanwar wrote:
Hello,

I am trying to build Bowtie2 index for hg19.
I a using Docker image of Galaxy.

Computer I am using have 12 cores and 64 GB of RAM.

Error I am getting is:

Fatal error: Exit code 1 ()
Building a SMALL index
Could not allocate space for a joined string of 2897310462 elements.
Switching to a packed string representation.
Building a SMALL index
Could not find approrpiate bmax/dcv settings for building this index.
Already tried a packed string representation.
Please try indexing this reference on a computer with more memory.
Error: Encountered internal Bowtie 2 exception (#1)
Command: bowtie2-build --wrapper basic-0
/export/galaxy-central/database/job_working_directory/000/113/dataset_80_files/hg19
fa.fa hg19 fa
Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
Error building index.

Please help!

Thank you,
Deepak



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bowtie2 index building

Deepak Tanwar
Also, I am running this container on a Mac Pro using Kitematic and I have docker preference set to use 12 cores and 64 GB RAM.

Thanks,
Deepak


On Aug 11, 2017, at 9:54 AM, Deepak Tanwar <[hidden email]> wrote:

Hi Björn,

If I do docker stats:
CONTAINER           CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
dd4da10c0a54        6.16%               690.9MiB / 995.8MiB   69.38%              158kB / 785kB       11.4GB / 81MB       106

And, after ssh into docker machine:

Filesystem                Size      Used Available Use% Mounted on
tmpfs                   896.2M    186.6M    709.6M  21% /
tmpfs                   497.9M    476.0K    497.4M   0% /dev/shm
/dev/sda1               282.7G     14.9G    253.2G   6% /mnt/sda1
cgroup                  497.9M         0    497.9M   0% /sys/fs/cgroup
Users                   931.0G    683.8G    247.2G  73% /Users
/dev/sda1               282.7G     14.9G    253.2G   6% /mnt/sda1/var/lib/docker/aufs

Best regards,
Deepak


On Aug 11, 2017, at 9:39 AM, Björn Grüning <[hidden email]> wrote:

Hi Deepak,

how are you running this Container?
Can you check how many memory you have inside the container?

Cheers,
Bjoern

On 11.08.2017 15:35, Deepak Tanwar wrote:
Hello,

I am trying to build Bowtie2 index for hg19.
I a using Docker image of Galaxy.

Computer I am using have 12 cores and 64 GB of RAM.

Error I am getting is:

Fatal error: Exit code 1 ()
Building a SMALL index
Could not allocate space for a joined string of 2897310462 elements.
Switching to a packed string representation.
Building a SMALL index
Could not find approrpiate bmax/dcv settings for building this index.
Already tried a packed string representation.
Please try indexing this reference on a computer with more memory.
Error: Encountered internal Bowtie 2 exception (#1)
Command: bowtie2-build --wrapper basic-0
/export/galaxy-central/database/job_working_directory/000/113/dataset_80_files/hg19
fa.fa hg19 fa
Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
Error building index.

Please help!

Thank you,
Deepak



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bowtie2 index building

Björn Grüning-3
Hi,

it seems that your Docker Box only gets 1GB :(

please re-check the configs.
Sorry to be not more helpful!
Bjoern

On 11.08.2017 15:55, Deepak Tanwar wrote:

> Also, I am running this container on a Mac Pro using Kitematic and I
> have docker preference set to use 12 cores and 64 GB RAM.
>
> Thanks,
> Deepak
>
>
>> On Aug 11, 2017, at 9:54 AM, Deepak Tanwar <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>> Hi Björn,
>>
>> If I do docker stats:
>> CONTAINER           CPU %               MEM USAGE / LIMIT     MEM %  
>>             NET I/O             BLOCK I/O           PIDS
>> dd4da10c0a54        6.16%               690.9MiB / 995.8MiB   69.38%  
>>            158kB / 785kB       11.4GB / 81MB       106
>>
>> And, after ssh into docker machine:
>>
>> Filesystem                Size      Used Available Use% Mounted on
>> tmpfs                   896.2M    186.6M    709.6M  21% /
>> tmpfs                   497.9M    476.0K    497.4M   0% /dev/shm
>> /dev/sda1               282.7G     14.9G    253.2G   6% /mnt/sda1
>> cgroup                  497.9M         0    497.9M   0% /sys/fs/cgroup
>> Users                   931.0G    683.8G    247.2G  73% /Users
>> /dev/sda1               282.7G     14.9G    253.2G   6%
>> /mnt/sda1/var/lib/docker/aufs
>>
>> Best regards,
>> Deepak
>>
>>
>>> On Aug 11, 2017, at 9:39 AM, Björn Grüning <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>> Hi Deepak,
>>>
>>> how are you running this Container?
>>> Can you check how many memory you have inside the container?
>>>
>>> Cheers,
>>> Bjoern
>>>
>>> On 11.08.2017 15:35, Deepak Tanwar wrote:
>>>> Hello,
>>>>
>>>> I am trying to build Bowtie2 index for hg19.
>>>> I a using Docker image of Galaxy.
>>>>
>>>> Computer I am using have 12 cores and 64 GB of RAM.
>>>>
>>>> Error I am getting is:
>>>>
>>>> Fatal error: Exit code 1 ()
>>>> Building a SMALL index
>>>> Could not allocate space for a joined string of 2897310462 elements.
>>>> Switching to a packed string representation.
>>>> Building a SMALL index
>>>> Could not find approrpiate bmax/dcv settings for building this index.
>>>> Already tried a packed string representation.
>>>> Please try indexing this reference on a computer with more memory.
>>>> Error: Encountered internal Bowtie 2 exception (#1)
>>>> Command: bowtie2-build --wrapper basic-0
>>>> /export/galaxy-central/database/job_working_directory/000/113/dataset_80_files/hg19
>>>> fa.fa hg19 fa
>>>> Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
>>>> Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
>>>> Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
>>>> Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
>>>> Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
>>>> Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
>>>> Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
>>>> Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
>>>> Error building index.
>>>>
>>>> Please help!
>>>>
>>>> Thank you,
>>>> Deepak
>>>>
>>>>
>>>>
>>>> ___________________________________________________________
>>>> Please keep all replies on the list by using "reply all"
>>>> in your mail client.  To manage your subscriptions to this
>>>> and other Galaxy lists, please use the interface at:
>>>>  https://lists.galaxyproject.org/
>>>>
>>>> To search Galaxy mailing lists use the unified search at:
>>>>  http://galaxyproject.org/search/
>>>>
>>
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bowtie2 index building

Deepak Tanwar
Hi Björn,

That’s right! It was quiet strange. I need to extend the memory and cores usage via command line.

Thank you so much!

Deepak


> On Aug 11, 2017, at 10:00, Björn Grüning <[hidden email]> wrote:
>
> Hi,
>
> it seems that your Docker Box only gets 1GB :(
>
> please re-check the configs.
> Sorry to be not more helpful!
> Bjoern
>
> On 11.08.2017 15:55, Deepak Tanwar wrote:
>> Also, I am running this container on a Mac Pro using Kitematic and I
>> have docker preference set to use 12 cores and 64 GB RAM.
>>
>> Thanks,
>> Deepak
>>
>>
>>> On Aug 11, 2017, at 9:54 AM, Deepak Tanwar <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>> Hi Björn,
>>>
>>> If I do docker stats:
>>> CONTAINER           CPU %               MEM USAGE / LIMIT     MEM %  
>>>            NET I/O             BLOCK I/O           PIDS
>>> dd4da10c0a54        6.16%               690.9MiB / 995.8MiB   69.38%  
>>>           158kB / 785kB       11.4GB / 81MB       106
>>>
>>> And, after ssh into docker machine:
>>>
>>> Filesystem                Size      Used Available Use% Mounted on
>>> tmpfs                   896.2M    186.6M    709.6M  21% /
>>> tmpfs                   497.9M    476.0K    497.4M   0% /dev/shm
>>> /dev/sda1               282.7G     14.9G    253.2G   6% /mnt/sda1
>>> cgroup                  497.9M         0    497.9M   0% /sys/fs/cgroup
>>> Users                   931.0G    683.8G    247.2G  73% /Users
>>> /dev/sda1               282.7G     14.9G    253.2G   6%
>>> /mnt/sda1/var/lib/docker/aufs
>>>
>>> Best regards,
>>> Deepak
>>>
>>>
>>>> On Aug 11, 2017, at 9:39 AM, Björn Grüning <[hidden email]
>>>> <mailto:[hidden email]>> wrote:
>>>>
>>>> Hi Deepak,
>>>>
>>>> how are you running this Container?
>>>> Can you check how many memory you have inside the container?
>>>>
>>>> Cheers,
>>>> Bjoern
>>>>
>>>> On 11.08.2017 15:35, Deepak Tanwar wrote:
>>>>> Hello,
>>>>>
>>>>> I am trying to build Bowtie2 index for hg19.
>>>>> I a using Docker image of Galaxy.
>>>>>
>>>>> Computer I am using have 12 cores and 64 GB of RAM.
>>>>>
>>>>> Error I am getting is:
>>>>>
>>>>> Fatal error: Exit code 1 ()
>>>>> Building a SMALL index
>>>>> Could not allocate space for a joined string of 2897310462 elements.
>>>>> Switching to a packed string representation.
>>>>> Building a SMALL index
>>>>> Could not find approrpiate bmax/dcv settings for building this index.
>>>>> Already tried a packed string representation.
>>>>> Please try indexing this reference on a computer with more memory.
>>>>> Error: Encountered internal Bowtie 2 exception (#1)
>>>>> Command: bowtie2-build --wrapper basic-0
>>>>> /export/galaxy-central/database/job_working_directory/000/113/dataset_80_files/hg19
>>>>> fa.fa hg19 fa
>>>>> Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
>>>>> Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
>>>>> Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
>>>>> Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
>>>>> Deleting "hg19 fa.3.bt2" file written during aborted indexing attempt.
>>>>> Deleting "hg19 fa.4.bt2" file written during aborted indexing attempt.
>>>>> Deleting "hg19 fa.1.bt2" file written during aborted indexing attempt.
>>>>> Deleting "hg19 fa.2.bt2" file written during aborted indexing attempt.
>>>>> Error building index.
>>>>>
>>>>> Please help!
>>>>>
>>>>> Thank you,
>>>>> Deepak
>>>>>
>>>>>
>>>>>
>>>>> ___________________________________________________________
>>>>> Please keep all replies on the list by using "reply all"
>>>>> in your mail client.  To manage your subscriptions to this
>>>>> and other Galaxy lists, please use the interface at:
>>>>> https://lists.galaxyproject.org/
>>>>>
>>>>> To search Galaxy mailing lists use the unified search at:
>>>>> http://galaxyproject.org/search/
>>>>>
>>>
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>> https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>> http://galaxyproject.org/search/
>>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Loading...