Pulling feature associated sequence data using web service api 'getSequence'

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Pulling feature associated sequence data using web service api 'getSequence'

Henderson, Louie

Dear Web Apollo developers,


I am trying to write a command line script to pull sequence data associated with user annotated features. To achieve this I've

been attempting to use the web service api functions, specifically 'getSequence'. However, I cannot get it to return a sensible

output. Other api functions (e.g. getFeature) provide readable errors given the same inputs suggesting some issue with this

specific api. Furthermore, the api examples such as getFeatures.sh appear to function.



Here is the current shell script I have been attempting to run:


#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
track=$5
features=$6

# First login to Apollo
curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences based on features
curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organism':'${organism}', 'track': '${track}', 'features':{uniquename':'${features}'}}" ${url}/annotationEditor/getSequence



As a novice to both json and Web Apollo, advice on how best to proceed and whether an alternate function might be better suited

would be appreciated!


Best wishes,


Louie Henderson






This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.

Reply | Threaded
Open this post in threaded view
|

Re: Pulling feature associated sequence data using web service api 'getSequence'

nathandunn

I have a few ideas.  

Are you trying to get the CDS / CDNA  / Protein on the feature or are you trying to pull a set genomic region.  Also, I’m unsure if you have the name or the unique_name.  

If you are on master / 2.0.8, you might try using Sequence Services:



I have no idea why I have a sequence_name in there, so if you don’t know that I can add another version. 



Nathan


On Nov 29, 2017, at 10:34 AM, Henderson, Louie <[hidden email]> wrote:

Dear Web Apollo developers,

I am trying to write a command line script to pull sequence data associated with user annotated features. To achieve this I've
been attempting to use the web service api functions, specifically 'getSequence'. However, I cannot get it to return a sensible
output. Other api functions (e.g. getFeature) provide readable errors given the same inputs suggesting some issue with this
specific api. Furthermore, the api examples such as getFeatures.sh appear to function.


Here is the current shell script I have been attempting to run:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
track=$5
features=$6

# First login to Apollo
curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences based on features
curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organism':'${organism}', 'track': '${track}', 'features':{uniquename':'${features}'}}" ${url}/annotationEditor/getSequence


As a novice to both json and Web Apollo, advice on how best to proceed and whether an alternate function might be better suited
would be appreciated!

Best wishes,

Louie Henderson




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 





This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.

Reply | Threaded
Open this post in threaded view
|

Re: Pulling feature associated sequence data using web service api 'getSequence'

Henderson, Louie

Hello Nathan,


Thanks for the reply! Ideally we would like to pull out the genomic region to begin with.


I've upgrade our installation to 2.0.8 in an attempt to use the new Sequence Services api however I still am having little success.

I was briefly able to interface with the database with getFeatures (though that itself is providing errors now) so have the unique

feature name to hand.


When I navigate to the api locations via a web browser (i.e. http://localhost:8080/apollo-2.0.8/annotationEditor/getFeatures) the

getFeatures and a few other api functions show information, whereas the getSequence api appears not to. Could there be some

error with the getSequence api itself?


Regarding the sequenceByName api, I'm receiving errors stating that the resources are unavailible -


HTTP Status 404 - &quot;/WEB-INF/grails-app/views/sequence/sequenceByName.gsp&quot; not found.


The code I'm using to generate this particular error is:


#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
seqtype=$5
sequence=$5
features=$6
ignorecache=$8

# First login to Apollo
echo curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences
echo curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}

curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}


I believe my inputs are correct however am now doubting everything!


Best wishes,


Louie Henderson



From: [hidden email] <[hidden email]> on behalf of Nathan Dunn <[hidden email]>
Sent: 30 November 2017 03:05:49
To: [hidden email]
Subject: Re: [apollo] Pulling feature associated sequence data using web service api 'getSequence'
 

I have a few ideas.  

Are you trying to get the CDS / CDNA  / Protein on the feature or are you trying to pull a set genomic region.  Also, I’m unsure if you have the name or the unique_name.  

If you are on master / 2.0.8, you might try using Sequence Services:



I have no idea why I have a sequence_name in there, so if you don’t know that I can add another version. 



Nathan


On Nov 29, 2017, at 10:34 AM, Henderson, Louie <[hidden email]> wrote:

Dear Web Apollo developers,

I am trying to write a command line script to pull sequence data associated with user annotated features. To achieve this I've
been attempting to use the web service api functions, specifically 'getSequence'. However, I cannot get it to return a sensible
output. Other api functions (e.g. getFeature) provide readable errors given the same inputs suggesting some issue with this
specific api. Furthermore, the api examples such as getFeatures.sh appear to function.


Here is the current shell script I have been attempting to run:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
track=$5
features=$6

# First login to Apollo
curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences based on features
curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organism':'${organism}', 'track': '${track}', 'features':{uniquename':'${features}'}}" ${url}/annotationEditor/getSequence


As a novice to both json and Web Apollo, advice on how best to proceed and whether an alternate function might be better suited
would be appreciated!

Best wishes,

Louie Henderson




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 





This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.

Reply | Threaded
Open this post in threaded view
|

Re: Pulling feature associated sequence data using web service api 'getSequence'

nathandunn

The easiest way is to make your organism data public and then just access it within the URL or via curl:




Nathan


On Dec 6, 2017, at 6:37 AM, Henderson, Louie <[hidden email]> wrote:

Hello Nathan,

Thanks for the reply! Ideally we would like to pull out the genomic region to begin with. 

I've upgrade our installation to 2.0.8 in an attempt to use the new Sequence Services api however I still am having little success. 
I was briefly able to interface with the database with getFeatures (though that itself is providing errors now) so have the unique 
feature name to hand. 

When I navigate to the api locations via a web browser (i.e. http://localhost:8080/apollo-2.0.8/annotationEditor/getFeatures) the 
getFeatures and a few other api functions show information, whereas the getSequence api appears not to. Could there be some
error with the getSequence api itself?

Regarding the sequenceByName api, I'm receiving errors stating that the resources are unavailible -

HTTP Status 404 - &quot;/WEB-INF/grails-app/views/sequence/sequenceByName.gsp&quot; not found.

The code I'm using to generate this particular error is:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
seqtype=$5
sequence=$5
features=$6
ignorecache=$8

# First login to Apollo
echo curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences 
echo curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}

curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}


I believe my inputs are correct however am now doubting everything!

Best wishes,

Louie Henderson


From: [hidden email] <[hidden email]> on behalf of Nathan Dunn <[hidden email]>
Sent: 30 November 2017 03:05:49
To: [hidden email]
Subject: Re: [apollo] Pulling feature associated sequence data using web service api 'getSequence'
 

I have a few ideas.  

Are you trying to get the CDS / CDNA  / Protein on the feature or are you trying to pull a set genomic region.  Also, I’m unsure if you have the name or the unique_name.  

If you are on master / 2.0.8, you might try using Sequence Services:


<Screen Shot 2017-11-29 at 7.04.25 PM.png>

I have no idea why I have a sequence_name in there, so if you don’t know that I can add another version. 



Nathan


On Nov 29, 2017, at 10:34 AM, Henderson, Louie <[hidden email]> wrote:

Dear Web Apollo developers,

I am trying to write a command line script to pull sequence data associated with user annotated features. To achieve this I've
been attempting to use the web service api functions, specifically 'getSequence'. However, I cannot get it to return a sensible
output. Other api functions (e.g. getFeature) provide readable errors given the same inputs suggesting some issue with this
specific api. Furthermore, the api examples such as getFeatures.sh appear to function.


Here is the current shell script I have been attempting to run:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
track=$5
features=$6

# First login to Apollo
curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences based on features
curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organism':'${organism}', 'track': '${track}', 'features':{uniquename':'${features}'}}" ${url}/annotationEditor/getSequence


As a novice to both json and Web Apollo, advice on how best to proceed and whether an alternate function might be better suited
would be appreciated!

Best wishes,

Louie Henderson




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 





This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.

Reply | Threaded
Open this post in threaded view
|

Re: Pulling feature associated sequence data using web service api 'getSequence'

Henderson, Louie

Hello Nathan,


Thanks for your help so far!


I've managed to pull the required data for a single organism (my input was slightly off, your example was helpful in correcting it).


However, to automate the process fully, I'll need both the unique feature names and sequence names for each organism. I previously

mentioned a foray into getFeatures.sh. I'm having issues getting it to work again, currently its outputting this regardless of input:


"problem getting features: java.lang.NullPointerException: Cannot get property 'name' on null object"


Is there another way to access annotated features? Furthermore, is there a way to list all sequence name information for a given organism?


(The ultimate aim is to be able to allow a user to pull genomic data associated with a given gene from every organism in our database

for tree building)


Best wishes,


Louie Henderson



From: [hidden email] <[hidden email]> on behalf of Nathan Dunn <[hidden email]>
Sent: 06 December 2017 17:56:30
To: [hidden email]
Subject: Re: [apollo] Pulling feature associated sequence data using web service api 'getSequence'
 

The easiest way is to make your organism data public and then just access it within the URL or via curl:




Nathan


On Dec 6, 2017, at 6:37 AM, Henderson, Louie <[hidden email]> wrote:

Hello Nathan,

Thanks for the reply! Ideally we would like to pull out the genomic region to begin with. 

I've upgrade our installation to 2.0.8 in an attempt to use the new Sequence Services api however I still am having little success. 
I was briefly able to interface with the database with getFeatures (though that itself is providing errors now) so have the unique 
feature name to hand. 

When I navigate to the api locations via a web browser (i.e. http://localhost:8080/apollo-2.0.8/annotationEditor/getFeatures) the 
getFeatures and a few other api functions show information, whereas the getSequence api appears not to. Could there be some
error with the getSequence api itself?

Regarding the sequenceByName api, I'm receiving errors stating that the resources are unavailible -

HTTP Status 404 - &quot;/WEB-INF/grails-app/views/sequence/sequenceByName.gsp&quot; not found.

The code I'm using to generate this particular error is:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
seqtype=$5
sequence=$5
features=$6
ignorecache=$8

# First login to Apollo
echo curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences 
echo curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}

curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}


I believe my inputs are correct however am now doubting everything!

Best wishes,

Louie Henderson


From: [hidden email] <[hidden email]> on behalf of Nathan Dunn <[hidden email]>
Sent: 30 November 2017 03:05:49
To: [hidden email]
Subject: Re: [apollo] Pulling feature associated sequence data using web service api 'getSequence'
 

I have a few ideas.  

Are you trying to get the CDS / CDNA  / Protein on the feature or are you trying to pull a set genomic region.  Also, I’m unsure if you have the name or the unique_name.  

If you are on master / 2.0.8, you might try using Sequence Services:


<Screen Shot 2017-11-29 at 7.04.25 PM.png>

I have no idea why I have a sequence_name in there, so if you don’t know that I can add another version. 



Nathan


On Nov 29, 2017, at 10:34 AM, Henderson, Louie <[hidden email]> wrote:

Dear Web Apollo developers,

I am trying to write a command line script to pull sequence data associated with user annotated features. To achieve this I've
been attempting to use the web service api functions, specifically 'getSequence'. However, I cannot get it to return a sensible
output. Other api functions (e.g. getFeature) provide readable errors given the same inputs suggesting some issue with this
specific api. Furthermore, the api examples such as getFeatures.sh appear to function.


Here is the current shell script I have been attempting to run:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
track=$5
features=$6

# First login to Apollo
curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences based on features
curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organism':'${organism}', 'track': '${track}', 'features':{uniquename':'${features}'}}" ${url}/annotationEditor/getSequence


As a novice to both json and Web Apollo, advice on how best to proceed and whether an alternate function might be better suited
would be appreciated!

Best wishes,

Louie Henderson




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 





This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.

Reply | Threaded
Open this post in threaded view
|

Re: Pulling feature associated sequence data using web service api 'getSequence'

nathandunn

There are a few ways to get all of the annotated features for an organism.   Do you want these as GFF3, FASTA, or as a JSON object? 

For GFF3 / FASTA I would use the get_gff3.groovy method: https://github.com/GMOD/Apollo/blob/master/docs/web_services/examples/groovy/get_gff3.groovy which you should have checked out when you closed Apollo:

./get_gff3.groovy -username [hidden email] -password demo -organism Honeybee -output local.gff -url http://demo.genomearchitect.org/Apollo2 

This gets all of the annotations as GFF3 for the organism Honeybee from our demo server.

./get_fasta.groovy -username [hidden email] -password demo -organism Honeybee -output local.fa -url http://demo.genomearchitect.org/Apollo2 -seqtype cdna


There are many ways to install groovy.  I like sdkman: http://sdkman.io/install.html  (then you do sdk install groovy), but anything should work.

To get it as JSON, you would emulate what Apollo does.  I’m working on a script that does this properly as what was in there for getFeatures was a bit confusing.  I’ll have that for you momentarily. 

Nathan



On Dec 7, 2017, at 11:33 AM, Henderson, Louie <[hidden email]> wrote:

Hello Nathan,

Thanks for your help so far! 

I've managed to pull the required data for a single organism (my input was slightly off, your example was helpful in correcting it).

However, to automate the process fully, I'll need both the unique feature names and sequence names for each organism. I previously 
mentioned a foray into getFeatures.sh. I'm having issues getting it to work again, currently its outputting this regardless of input:

"problem getting features: java.lang.NullPointerException: Cannot get property 'name' on null object"

Is there another way to access annotated features? Furthermore, is there a way to list all sequence name information for a given organism?

(The ultimate aim is to be able to allow a user to pull genomic data associated with a given gene from every organism in our database
for tree building)

Best wishes,

Louie Henderson


From: [hidden email] <[hidden email]> on behalf of Nathan Dunn <[hidden email]>
Sent: 06 December 2017 17:56:30
To: [hidden email]
Subject: Re: [apollo] Pulling feature associated sequence data using web service api 'getSequence'
 

The easiest way is to make your organism data public and then just access it within the URL or via curl:



<Screen Shot 2017-12-06 at 9.55.08 AM.png>

Nathan


On Dec 6, 2017, at 6:37 AM, Henderson, Louie <[hidden email]> wrote:

Hello Nathan,

Thanks for the reply! Ideally we would like to pull out the genomic region to begin with. 

I've upgrade our installation to 2.0.8 in an attempt to use the new Sequence Services api however I still am having little success. 
I was briefly able to interface with the database with getFeatures (though that itself is providing errors now) so have the unique 
feature name to hand. 

When I navigate to the api locations via a web browser (i.e. http://localhost:8080/apollo-2.0.8/annotationEditor/getFeatures) the 
getFeatures and a few other api functions show information, whereas the getSequence api appears not to. Could there be some
error with the getSequence api itself?

Regarding the sequenceByName api, I'm receiving errors stating that the resources are unavailible -

HTTP Status 404 - &quot;/WEB-INF/grails-app/views/sequence/sequenceByName.gsp&quot; not found.

The code I'm using to generate this particular error is:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
seqtype=$5
sequence=$5
features=$6
ignorecache=$8

# First login to Apollo
echo curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences 
echo curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}

curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}


I believe my inputs are correct however am now doubting everything!

Best wishes,

Louie Henderson


From: [hidden email] <[hidden email]> on behalf of Nathan Dunn <[hidden email]>
Sent: 30 November 2017 03:05:49
To: [hidden email]
Subject: Re: [apollo] Pulling feature associated sequence data using web service api 'getSequence'
 

I have a few ideas.  

Are you trying to get the CDS / CDNA  / Protein on the feature or are you trying to pull a set genomic region.  Also, I’m unsure if you have the name or the unique_name.  

If you are on master / 2.0.8, you might try using Sequence Services:


<Screen Shot 2017-11-29 at 7.04.25 PM.png>

I have no idea why I have a sequence_name in there, so if you don’t know that I can add another version. 



Nathan


On Nov 29, 2017, at 10:34 AM, Henderson, Louie <[hidden email]> wrote:

Dear Web Apollo developers,

I am trying to write a command line script to pull sequence data associated with user annotated features. To achieve this I've
been attempting to use the web service api functions, specifically 'getSequence'. However, I cannot get it to return a sensible
output. Other api functions (e.g. getFeature) provide readable errors given the same inputs suggesting some issue with this
specific api. Furthermore, the api examples such as getFeatures.sh appear to function.


Here is the current shell script I have been attempting to run:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
track=$5
features=$6

# First login to Apollo
curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences based on features
curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organism':'${organism}', 'track': '${track}', 'features':{uniquename':'${features}'}}" ${url}/annotationEditor/getSequence


As a novice to both json and Web Apollo, advice on how best to proceed and whether an alternate function might be better suited
would be appreciated!

Best wishes,

Louie Henderson




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 





This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.

Reply | Threaded
Open this post in threaded view
|

Re: Pulling feature associated sequence data using web service api 'getSequence'

nathandunn

I updated getFeatures.sh:

 ./getFeatures.sh http://demo.genomearchitect.org/Apollo2 [hidden email] demo Honeybee Group1.10


Generates this:

curl -H Content-Type:application/json --data { 'organism':'Honeybee','operation': 'get_features', 'track': 'Group1.10','username':'[hidden email]','password':'demo'} http://demo.genomearchitect.org/Apollo2/annotationEditor/getFeatures

https://github.com/GMOD/Apollo/blob/master/docs/web_services/examples/shell/getFeatures.sh


I think you were close originally, but something was slightly off. 

This will pull the JSON that is used within JBrowse /Apollo from the server. 



Nathan

On Dec 7, 2017, at 4:20 PM, Nathan Dunn <[hidden email]> wrote:


There are a few ways to get all of the annotated features for an organism.   Do you want these as GFF3, FASTA, or as a JSON object? 

For GFF3 / FASTA I would use the get_gff3.groovy method: https://github.com/GMOD/Apollo/blob/master/docs/web_services/examples/groovy/get_gff3.groovy which you should have checked out when you closed Apollo:

./get_gff3.groovy -username [hidden email] -password demo -organism Honeybee -output local.gff -url http://demo.genomearchitect.org/Apollo2 

This gets all of the annotations as GFF3 for the organism Honeybee from our demo server.

./get_fasta.groovy -username [hidden email] -password demo -organism Honeybee -output local.fa -url http://demo.genomearchitect.org/Apollo2 -seqtype cdna


There are many ways to install groovy.  I like sdkman: http://sdkman.io/install.html  (then you do sdk install groovy), but anything should work.

To get it as JSON, you would emulate what Apollo does.  I’m working on a script that does this properly as what was in there for getFeatures was a bit confusing.  I’ll have that for you momentarily. 

Nathan



On Dec 7, 2017, at 11:33 AM, Henderson, Louie <[hidden email]> wrote:

Hello Nathan,

Thanks for your help so far! 

I've managed to pull the required data for a single organism (my input was slightly off, your example was helpful in correcting it).

However, to automate the process fully, I'll need both the unique feature names and sequence names for each organism. I previously 
mentioned a foray into getFeatures.sh. I'm having issues getting it to work again, currently its outputting this regardless of input:

"problem getting features: java.lang.NullPointerException: Cannot get property 'name' on null object"

Is there another way to access annotated features? Furthermore, is there a way to list all sequence name information for a given organism?

(The ultimate aim is to be able to allow a user to pull genomic data associated with a given gene from every organism in our database
for tree building)

Best wishes,

Louie Henderson


From: [hidden email] <[hidden email]> on behalf of Nathan Dunn <[hidden email]>
Sent: 06 December 2017 17:56:30
To: [hidden email]
Subject: Re: [apollo] Pulling feature associated sequence data using web service api 'getSequence'
 

The easiest way is to make your organism data public and then just access it within the URL or via curl:



<Screen Shot 2017-12-06 at 9.55.08 AM.png>

Nathan


On Dec 6, 2017, at 6:37 AM, Henderson, Louie <[hidden email]> wrote:

Hello Nathan,

Thanks for the reply! Ideally we would like to pull out the genomic region to begin with. 

I've upgrade our installation to 2.0.8 in an attempt to use the new Sequence Services api however I still am having little success. 
I was briefly able to interface with the database with getFeatures (though that itself is providing errors now) so have the unique 
feature name to hand. 

When I navigate to the api locations via a web browser (i.e. http://localhost:8080/apollo-2.0.8/annotationEditor/getFeatures) the 
getFeatures and a few other api functions show information, whereas the getSequence api appears not to. Could there be some
error with the getSequence api itself?

Regarding the sequenceByName api, I'm receiving errors stating that the resources are unavailible -

HTTP Status 404 - &quot;/WEB-INF/grails-app/views/sequence/sequenceByName.gsp&quot; not found.

The code I'm using to generate this particular error is:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
seqtype=$5
sequence=$5
features=$6
ignorecache=$8

# First login to Apollo
echo curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences 
echo curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}

curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organismString':'${organism}', 'sequence': '${sequence}', 'featureName':'${features}', 'seqtype':'${seqtype}', 'ignorecache':'${ignorecache}}" ${url}/sequence/${organism}/${sequence}/${features}.${seqtype}?ignoreCache=${ignorecache}


I believe my inputs are correct however am now doubting everything!

Best wishes,

Louie Henderson


From: [hidden email] <[hidden email]> on behalf of Nathan Dunn <[hidden email]>
Sent: 30 November 2017 03:05:49
To: [hidden email]
Subject: Re: [apollo] Pulling feature associated sequence data using web service api 'getSequence'
 

I have a few ideas.  

Are you trying to get the CDS / CDNA  / Protein on the feature or are you trying to pull a set genomic region.  Also, I’m unsure if you have the name or the unique_name.  

If you are on master / 2.0.8, you might try using Sequence Services:


<Screen Shot 2017-11-29 at 7.04.25 PM.png>

I have no idea why I have a sequence_name in there, so if you don’t know that I can add another version. 



Nathan


On Nov 29, 2017, at 10:34 AM, Henderson, Louie <[hidden email]> wrote:

Dear Web Apollo developers,

I am trying to write a command line script to pull sequence data associated with user annotated features. To achieve this I've
been attempting to use the web service api functions, specifically 'getSequence'. However, I cannot get it to return a sensible
output. Other api functions (e.g. getFeature) provide readable errors given the same inputs suggesting some issue with this
specific api. Furthermore, the api examples such as getFeatures.sh appear to function.


Here is the current shell script I have been attempting to run:

#!/bin/bash

url=$1
username=$2
password=$3
organism=$4
track=$5
features=$6

# First login to Apollo
curl -b login_cookies.txt -c login_cookies.txt -H "Content-Type:application/json" -d "{'username': '${username}', 'password': '${password}'}" ${url}/Login?operation=login

# Then request sequences based on features
curl -b login_cookies.txt -c login_cookies.txt -e "${url}" --data "{'organism':'${organism}', 'track': '${track}', 'features':{uniquename':'${features}'}}" ${url}/annotationEditor/getSequence


As a novice to both json and Web Apollo, advice on how best to proceed and whether an alternate function might be better suited
would be appreciated!

Best wishes,

Louie Henderson




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 




This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 






This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [hidden email] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.