Re: Custom Datasource Error

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Custom Datasource Error

Julie Sullivan-2
Hi Justin!

Here are the docs on versioning your sources:

https://intermine.readthedocs.io/en/latest/database/data-sources/versions/

There is an "InterMine version" set, which for you is 4.0.1 (I think?).
This is what the sources default to, and why the build is looking for
that version. You can either:

option A: Set your custom source to be version 4.0.1
option B: Set your custom source manually in project XML (e.g.
version="1.0.0")

Either will work.

1. version is set in the build gradle file, e.g.
https://github.com/intermine/humanmine-bio-sources/blob/master/build.gradle#L24
2. You can verify your JAR was created with the correct version by
looking in the maven directory:

 > ~/.m2/repository/org/intermine/bio-source-hpo $ ls
 > 4.0.0  maven-metadata-local.xml

What version is your dbsnp source set at now?

Julie


PS. I've CCed our public developers mailing list as others may have this
same issue!

https://lists.intermine.org/pipermail/dev/

On 24/07/2019 16:52, Le Tourneau, Justin J. wrote:

> Hi Julie!
>
> Dr. Christine Elsike and Shamim from my lab gave me your email
> concerning Intermine issues. I hope it’s okay to email you directly.
>
> I’ve installed and setup the latest version of Intermine without any
> issue (I’m trying to start to move our instances to the latest version).
> However, when I try to create a new datasource (a test for dbsnp data in
> this case) I get the error below. This is just after creating my
> ‘dbsnp-test’ source and following the directions from your documentation
> (https://intermine.readthedocs.io/en/latest/database/data-sources/custom/).
> The documentation is a bit confusing and unclear on versioning, but, I
> think, I do not need to worry as this is just a new test instance of the
> parser? I can just let it default to 4+ ?
>
> It seems to be looking for the source in my home directory and then
> remotely, is there a way to point it to the bio/sources directory?
>
> Any help would be appreciated,
>
> Thanks!
>
> -Justin
>
> project.xml:
>
> <source name="dbsnp-test" type="dbsnp-test">
>
> <property name="geneSource" value="Ensembl95"/>
>
> <property name="src.data.dir.includes" value="*.vcf"/>
>
> <property name="src.data.dir" location="/db/intermine/data/part_I"/>
>
> </source>
>
> [letourneaujj@intermineTest dbsnptest]$ cd
> /db/intermine/intermine/bio/sources/
>
> [letourneaujj@intermineTest dbsnptest]$ ./intermine-scripts/make_source
> dbsnp-test custom-file
>
> [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB
>
>> Configure project :
>
> Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
>
> cleaning
>
> deleting idresolver cache
>
> deleting datasources.xml
>
> deleting organisms.xml
>
> deleting publications.xml
>
> deleted log files
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
>
> Execution failed for task ':dbmodel:generateKeys'.
>
>> Could not resolve all files for configuration ':dbmodel:mergeSource'.
>
>     > Could not find any matches for
> org.intermine:bio-source-dbsnp-test:4+ as no versions of
> org.intermine:bio-source-dbsnp-test are available.
>
>       Searched in the following locations:
>
>          
> file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/maven-metadata.xml
>
>          
> file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/
>
>          
> https://jcenter.bintray.com/org/intermine/bio-source-testing/maven-metadata.xml
>
>           https://jcenter.bintray.com/org/intermine/bio-source-testing/
>
>          
> https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/maven-metadata.xml
>
>          
> https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/
>
>       Required by:
>
>           project :dbmodel
>
> * Try:
>
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output.
>
> * Get more help at https://help.gradle.org
>
> BUILD FAILED in 9s
>
> 10 actionable tasks: 8 executed, 2 up-to-date
>
_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Custom Datasource Error

Le Tourneau, Justin J.

Hey Julie,

 

Thanks for the quick reply. I have the version set in both places:

 

build.gradle

subprojects {

    group = 'org.intermine'

    version = '4.0.1'

 

project.xml

<source name="dbsnp-test" type="dbsnp-test" version=”4.0.1”>

 

 

There’s nothing being written to the maven directory:

 

[letourneaujj@intermineTest dbsnptest]$ ll ~/.m2/repository/

total 0       

 

 

and, for whatever reason, it’s still looking remotely for these resources??  This is the most confusing thing to me. Is there some way for your build system to look locally?

 

 

Sorry for being a pain, but I’m really scratching my head here. Following the directions verbatim from the documentation should work without any configuration changes, no?  

 

 

 

[letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB

 

> Configure project :

Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'

cleaning

deleting idresolver cache

deleting datasources.xml

deleting organisms.xml

deleting publications.xml

deleted log files

 

 

FAILURE: Build failed with an exception.

 

* What went wrong:

Execution failed for task ':dbmodel:generateKeys'.

> Could not resolve all files for configuration ':dbmodel:mergeSource'.

   > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.

     Required by:

         project :dbmodel

      > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.

         > Could not get resource 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test -4.0.1.pom'.

            > Could not GET 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test -4.0.1.pom'. Received status code 409 from server:

 

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

 

* Get more help at https://help.gradle.org

 

BUILD FAILED in 5s

10 actionable tasks: 7 executed, 3 up-to-date

 

 

 

 

 

On 7/25/19, 2:16 AM, "Julie Sullivan" <[hidden email]> wrote:

 

    Hi Justin!

   

    Here are the docs on versioning your sources:

   

    https://intermine.readthedocs.io/en/latest/database/data-sources/versions/

   

    There is an "InterMine version" set, which for you is 4.0.1 (I think?).

    This is what the sources default to, and why the build is looking for

    that version. You can either:

   

    option A: Set your custom source to be version 4.0.1

    option B: Set your custom source manually in project XML (e.g.

    version="1.0.0")

   

    Either will work.

   

    1. version is set in the build gradle file, e.g.

    https://github.com/intermine/humanmine-bio-sources/blob/master/build.gradle#L24

    2. You can verify your JAR was created with the correct version by

    looking in the maven directory:

   

     > ~/.m2/repository/org/intermine/bio-source-hpo $ ls

     > 4.0.0  maven-metadata-local.xml

   

    What version is your dbsnp source set at now?

   

    Julie

   

    

    PS. I've CCed our public developers mailing list as others may have this

    same issue!

   

    https://lists.intermine.org/pipermail/dev/

   

    On 24/07/2019 16:52, Le Tourneau, Justin J. wrote:

    > Hi Julie!

    >

    > Dr. Christine Elsike and Shamim from my lab gave me your email

    > concerning Intermine issues. I hope it’s okay to email you directly.

    >

    > I’ve installed and setup the latest version of Intermine without any

    > issue (I’m trying to start to move our instances to the latest version).

    > However, when I try to create a new datasource (a test for dbsnp data in

    > this case) I get the error below. This is just after creating my

    > ‘dbsnp-test’ source and following the directions from your documentation

    > (https://intermine.readthedocs.io/en/latest/database/data-sources/custom/).

    > The documentation is a bit confusing and unclear on versioning, but, I

    > think, I do not need to worry as this is just a new test instance of the

    > parser? I can just let it default to 4+ ?

    >

    > It seems to be looking for the source in my home directory and then

    > remotely, is there a way to point it to the bio/sources directory?

    >

    > Any help would be appreciated,

    >

    > Thanks!

    >

    > -Justin

    >

    > project.xml:

    >

    > <source name="dbsnp-test" type="dbsnp-test">

    >

    > <property name="geneSource" value="Ensembl95"/>

    >

    > <property name="src.data.dir.includes" value="*.vcf"/>

    >

    > <property name="src.data.dir" location="/db/intermine/data/part_I"/>

    >

    > </source>

    >

    > [letourneaujj@intermineTest dbsnptest]$ cd

    > /db/intermine/intermine/bio/sources/

    >

    > [letourneaujj@intermineTest dbsnptest]$ ./intermine-scripts/make_source

    > dbsnp-test custom-file

    >

    > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB

    >

    >> Configure project :

    >

    > Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'

    >

    > cleaning

    >

    > deleting idresolver cache

    >

    > deleting datasources.xml

    >

    > deleting organisms.xml

    >

    > deleting publications.xml

    >

    > deleted log files

    >

    > FAILURE: Build failed with an exception.

    >

    > * What went wrong:

    >

    > Execution failed for task ':dbmodel:generateKeys'.

    >

    >> Could not resolve all files for configuration ':dbmodel:mergeSource'.

    >

    >     > Could not find any matches for

    > org.intermine:bio-source-dbsnp-test:4+ as no versions of

    > org.intermine:bio-source-dbsnp-test are available.

    >

    >       Searched in the following locations:

    >

    >          

    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/maven-metadata.xml

    >

    >          

    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/

    >

    >          

    > https://jcenter.bintray.com/org/intermine/bio-source-testing/maven-metadata.xml

    >

    >           https://jcenter.bintray.com/org/intermine/bio-source-testing/

    >

    >          

    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/maven-metadata.xml

    >

    >          

    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/

    >

    >       Required by:

    >

    >           project :dbmodel

    >

    > * Try:

    >

    > Run with --stacktrace option to get the stack trace. Run with --info or

    > --debug option to get more log output.

    >

    > * Get more help at https://help.gradle.org

    >

    > BUILD FAILED in 9s

    >

    > 10 actionable tasks: 8 executed, 2 up-to-date

    >

    


_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Custom Datasource Error

Julie Sullivan-2
Hi Justin,

No problem at all!

HOW TO PUT YOUR SOURCE ON THE CLASSPATH

The build assumes that your code is compiled, in a JAR and on the
classpath. To do that, run this command:

        ~/bio/sources $ ./gradlew install

You should then see your JAR in your maven directory. If you make any
changes to your code, you will want to re-run that command -- to
recompile and make another JAR.

WHY MAVEN IS LOOKING IN THE REMOTE REPO

We have Maven configured to look locally then to look in Maven central.
We have to look both places for dependencies. Your source is local, but
most of your dependencies (e.g. apache-commons) will be in the remote
maven repo.

Here is where we set the repos:

https://github.com/intermine/flymine-bio-sources/blob/master/build.gradle#L37

         mavenLocal() <-- looks locally
         jcenter() <-- on failure, it will look in remote repo

So it checks locally first, fails to find your JAR. Then it looks in the
remote repo, and fails too. That's why you see that error.

I hope that helps!
Julie

On 25/07/2019 16:52, Le Tourneau, Justin J. wrote:

> Hey Julie,
>
> Thanks for the quick reply. I have the version set in both places:
>
> build.gradle
>
> subprojects {
>
>      group = 'org.intermine'
>
>      version = '4.0.1'
>
> project.xml
>
> <source name="dbsnp-test" type="dbsnp-test" version=”4.0.1”>
>
> There’s nothing being written to the maven directory:
>
> [letourneaujj@intermineTest dbsnptest]$ ll ~/.m2/repository/
>
> total 0
>
> and, for whatever reason, it’s still looking remotely for these
> resources??  This is the most confusing thing to me. Is there some way
> for your build system to look locally?
>
> Sorry for being a pain, but I’m really scratching my head here.
> Following the directions verbatim from the documentation should work
> without any configuration changes, no?
>
> [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB
>
>> Configure project :
>
> Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
>
> cleaning
>
> deleting idresolver cache
>
> deleting datasources.xml
>
> deleting organisms.xml
>
> deleting publications.xml
>
> deleted log files
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
>
> Execution failed for task ':dbmodel:generateKeys'.
>
>> Could not resolve all files for configuration ':dbmodel:mergeSource'.
>
>     > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.
>
>       Required by:
>
>           project :dbmodel
>
>        > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.
>
>           > Could not get resource
> 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test 
> -4.0.1.pom'.
>
>              > Could not GET
> 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test 
> -4.0.1.pom'. Received status code 409 from server:
>
> * Try:
>
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output.
>
> * Get more help at https://help.gradle.org
>
> BUILD FAILED in 5s
>
> 10 actionable tasks: 7 executed, 3 up-to-date
>
> On 7/25/19, 2:16 AM, "Julie Sullivan" <[hidden email]> wrote:
>
>      Hi Justin!
>
>      Here are the docs on versioning your sources:
>
>      https://intermine.readthedocs.io/en/latest/database/data-sources/versions/
>
>      There is an "InterMine version" set, which for you is 4.0.1 (I
> think?).
>
>      This is what the sources default to, and why the build is looking for
>
>      that version. You can either:
>
>      option A: Set your custom source to be version 4.0.1
>
>      option B: Set your custom source manually in project XML (e.g.
>
>      version="1.0.0")
>
>      Either will work.
>
>      1. version is set in the build gradle file, e.g.
>
>      https://github.com/intermine/humanmine-bio-sources/blob/master/build.gradle#L24
>
>      2. You can verify your JAR was created with the correct version by
>
>      looking in the maven directory:
>
>       > ~/.m2/repository/org/intermine/bio-source-hpo $ ls
>
>       > 4.0.0  maven-metadata-local.xml
>
>      What version is your dbsnp source set at now?
>
>      Julie
>
>      PS. I've CCed our public developers mailing list as others may have
> this
>
>      same issue!
>
>      https://lists.intermine.org/pipermail/dev/
>
>      On 24/07/2019 16:52, Le Tourneau, Justin J. wrote:
>
>      > Hi Julie!
>
>      >
>
>      > Dr. Christine Elsike and Shamim from my lab gave me your email
>
>      > concerning Intermine issues. I hope it’s okay to email you directly.
>
>      >
>
>      > I’ve installed and setup the latest version of Intermine without any
>
>      > issue (I’m trying to start to move our instances to the latest
> version).
>
>      > However, when I try to create a new datasource (a test for dbsnp
> data in
>
>      > this case) I get the error below. This is just after creating my
>
>      > ‘dbsnp-test’ source and following the directions from your
> documentation
>
>      >
> (https://intermine.readthedocs.io/en/latest/database/data-sources/custom/).
>
>      > The documentation is a bit confusing and unclear on versioning,
> but, I
>
>      > think, I do not need to worry as this is just a new test instance
> of the
>
>      > parser? I can just let it default to 4+ ?
>
>      >
>
>      > It seems to be looking for the source in my home directory and then
>
>      > remotely, is there a way to point it to the bio/sources directory?
>
>      >
>
>      > Any help would be appreciated,
>
>      >
>
>      > Thanks!
>
>      >
>
>      > -Justin
>
>      >
>
>      > project.xml:
>
>      >
>
>      > <source name="dbsnp-test" type="dbsnp-test">
>
>      >
>
>      > <property name="geneSource" value="Ensembl95"/>
>
>      >
>
>      > <property name="src.data.dir.includes" value="*.vcf"/>
>
>      >
>
>      > <property name="src.data.dir" location="/db/intermine/data/part_I"/>
>
>      >
>
>      > </source>
>
>      >
>
>      > [letourneaujj@intermineTest dbsnptest]$ cd
>
>      > /db/intermine/intermine/bio/sources/
>
>      >
>
>      > [letourneaujj@intermineTest dbsnptest]$
> ./intermine-scripts/make_source
>
>      > dbsnp-test custom-file
>
>      >
>
>      > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB
>
>      >
>
>      >> Configure project :
>
>      >
>
>      > Using properties file:
> '/home/letourneaujj/.intermine/dbsnptest.properties'
>
>      >
>
>      > cleaning
>
>      >
>
>      > deleting idresolver cache
>
>      >
>
>      > deleting datasources.xml
>
>      >
>
>      > deleting organisms.xml
>
>      >
>
>      > deleting publications.xml
>
>      >
>
>      > deleted log files
>
>      >
>
>      > FAILURE: Build failed with an exception.
>
>      >
>
>      > * What went wrong:
>
>      >
>
>      > Execution failed for task ':dbmodel:generateKeys'.
>
>      >
>
>      >> Could not resolve all files for configuration
> ':dbmodel:mergeSource'.
>
>      >
>
>      >     > Could not find any matches for
>
>      > org.intermine:bio-source-dbsnp-test:4+ as no versions of
>
>      > org.intermine:bio-source-dbsnp-test are available.
>
>      >
>
>      >       Searched in the following locations:
>
>      >
>
>      >
>
>      >
> file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/maven-metadata.xml
>
>      >
>
>      >
>
>      >
> file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/
>
>      >
>
>      >
>
>      >
> https://jcenter.bintray.com/org/intermine/bio-source-testing/maven-metadata.xml
>
>      >
>
>      >          
> https://jcenter.bintray.com/org/intermine/bio-source-testing/
>
>      >
>
>      >
>
>      >
> https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/maven-metadata.xml
>
>      >
>
>      >
>
>      >
> https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/
>
>      >
>
>      >       Required by:
>
>      >
>
>      >           project :dbmodel
>
>      >
>
>      > * Try:
>
>      >
>
>      > Run with --stacktrace option to get the stack trace. Run with
> --info or
>
>      > --debug option to get more log output.
>
>      >
>
>      > * Get more help at https://help.gradle.org
>
>      >
>
>      > BUILD FAILED in 9s
>
>      >
>
>      > 10 actionable tasks: 8 executed, 2 up-to-date
>
>      >
>
_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Custom Datasource Error

Le Tourneau, Justin J.

Thanks!

 

Running

 

[letourneaujj@intermineTest sources]$ /db/intermine/intermine/bio/sources/gradlew install

 

BUILD SUCCESSFUL in 19s

440 actionable tasks: 200 executed, 240 up-to-date

 

builds a /db/intermine/intermine/bio/sources/settings.gradle file that does not contain any of my custom sources (not sure if this is correct?).

 

I then tried creating a bio/sources directory in my home directory with the newly created parsers. I also added

 

maven {

      url ‘/db/intermine/bio/sources’

}

 

just to try adding a verbatim local source

 

 

I still received the same error: Maven is still looking at remote repositories. With the verbatim local source:

 

FAILURE: Build failed with an exception.

 

* What went wrong:

Execution failed for task ':dbmodel:generateKeys'.

> Could not resolve all files for configuration ':dbmodel:mergeSource'.

 

   > Could not find any matches for org.intermine:bio-source-testing:4+ as no versions of org.intermine:bio-source-testing are available.

     Searched in the following locations:

         file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-test/maven-metadata.xml

         file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-test/

         file:/db/intermine/intermine/bio/custom_sources/org/intermine/bio-source-dbsnp-test/maven-metadata.xml

         file:/db/intermine/intermine/bio/custom_sources/org/intermine/bio-source-dbsnp-test/

         https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-test/maven-metadata.xml

         https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-test/

         https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-test/maven-metadata.xml

         https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-test/

     Required by:

         project :dbmodel

 

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

 

* Get more help at https://help.gradle.org

 

Gradle / Maven doesn’t seem to ‘see’ any of these files?

 

 

Thanks for your help,

 

-Justin

 

 

 

On 7/25/19, 11:08 AM, "Julie Sullivan" <[hidden email]> wrote:

 

    Hi Justin,

   

    No problem at all!

   

    HOW TO PUT YOUR SOURCE ON THE CLASSPATH

   

    The build assumes that your code is compiled, in a JAR and on the

    classpath. To do that, run this command:

   

                ~/bio/sources $ ./gradlew install

   

    You should then see your JAR in your maven directory. If you make any

    changes to your code, you will want to re-run that command -- to

    recompile and make another JAR.

   

    WHY MAVEN IS LOOKING IN THE REMOTE REPO

   

    We have Maven configured to look locally then to look in Maven central.

    We have to look both places for dependencies. Your source is local, but

    most of your dependencies (e.g. apache-commons) will be in the remote

    maven repo.

   

    Here is where we set the repos:

   

    https://github.com/intermine/flymine-bio-sources/blob/master/build.gradle#L37

   

             mavenLocal() <-- looks locally

             jcenter() <-- on failure, it will look in remote repo

   

    So it checks locally first, fails to find your JAR. Then it looks in the

    remote repo, and fails too. That's why you see that error.

   

    I hope that helps!

    Julie

   

    On 25/07/2019 16:52, Le Tourneau, Justin J. wrote:

    > Hey Julie,

    >

    > Thanks for the quick reply. I have the version set in both places:

    >

    > build.gradle

    >

    > subprojects {

    >

    >      group = 'org.intermine'

    >

    >      version = '4.0.1'

    >

    > project.xml

    >

    > <source name="dbsnp-test" type="dbsnp-test" version=”4.0.1”>

    >

    > There’s nothing being written to the maven directory:

    >

    > [letourneaujj@intermineTest dbsnptest]$ ll ~/.m2/repository/

    >

    > total 0

    >

    > and, for whatever reason, it’s still looking remotely for these

    > resources??  This is the most confusing thing to me. Is there some way

    > for your build system to look locally?

    >

    > Sorry for being a pain, but I’m really scratching my head here.

    > Following the directions verbatim from the documentation should work

    > without any configuration changes, no?

    >

    > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB

    >

    >> Configure project :

    >

    > Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'

    >

    > cleaning

    >

    > deleting idresolver cache

    >

    > deleting datasources.xml

    >

    > deleting organisms.xml

    >

    > deleting publications.xml

    >

    > deleted log files

    >

    > FAILURE: Build failed with an exception.

    >

    > * What went wrong:

    >

    > Execution failed for task ':dbmodel:generateKeys'.

    >

    >> Could not resolve all files for configuration ':dbmodel:mergeSource'.

    >

    >     > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.

    >

    >       Required by:

    >

    >           project :dbmodel

    >

    >        > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.

    >

    >           > Could not get resource

    > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test

    > -4.0.1.pom'.

    >

    >              > Could not GET

    > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test

    > -4.0.1.pom'. Received status code 409 from server:

    >

    > * Try:

    >

    > Run with --stacktrace option to get the stack trace. Run with --info or

    > --debug option to get more log output.

    >

    > * Get more help at https://help.gradle.org

    >

    > BUILD FAILED in 5s

    >

    > 10 actionable tasks: 7 executed, 3 up-to-date

    >

    > On 7/25/19, 2:16 AM, "Julie Sullivan" <[hidden email]> wrote:

    >

    >      Hi Justin!

    >

    >      Here are the docs on versioning your sources:

    >

    >      https://intermine.readthedocs.io/en/latest/database/data-sources/versions/

    >

    >      There is an "InterMine version" set, which for you is 4.0.1 (I

    > think?).

    >

    >      This is what the sources default to, and why the build is looking for

    >

    >      that version. You can either:

    >

    >      option A: Set your custom source to be version 4.0.1

    >

    >      option B: Set your custom source manually in project XML (e.g.

    >

    >      version="1.0.0")

    >

    >      Either will work.

    >

    >      1. version is set in the build gradle file, e.g.

    >

    >      https://github.com/intermine/humanmine-bio-sources/blob/master/build.gradle#L24

    >

    >      2. You can verify your JAR was created with the correct version by

    >

    >      looking in the maven directory:

    >

    >       > ~/.m2/repository/org/intermine/bio-source-hpo $ ls

    >

    >       > 4.0.0  maven-metadata-local.xml

    >

    >      What version is your dbsnp source set at now?

    >

    >      Julie

    >

    >      PS. I've CCed our public developers mailing list as others may have

    > this

    >

    >      same issue!

    >

    >      https://lists.intermine.org/pipermail/dev/

    >

    >      On 24/07/2019 16:52, Le Tourneau, Justin J. wrote:

    >

    >      > Hi Julie!

    >

    >      >

    >

    >      > Dr. Christine Elsike and Shamim from my lab gave me your email

    >

    >      > concerning Intermine issues. I hope it’s okay to email you directly.

    >

    >      >

    >

    >      > I’ve installed and setup the latest version of Intermine without any

    >

    >      > issue (I’m trying to start to move our instances to the latest

    > version).

    >

    >      > However, when I try to create a new datasource (a test for dbsnp

    > data in

    >

    >      > this case) I get the error below. This is just after creating my

    >

    >      > ‘dbsnp-test’ source and following the directions from your

    > documentation

    >

    >      >

    > (https://intermine.readthedocs.io/en/latest/database/data-sources/custom/).

    >

    >      > The documentation is a bit confusing and unclear on versioning,

    > but, I

    >

    >      > think, I do not need to worry as this is just a new test instance

    > of the

    >

    >      > parser? I can just let it default to 4+ ?

    >

    >      >

    >

    >      > It seems to be looking for the source in my home directory and then

    >

    >      > remotely, is there a way to point it to the bio/sources directory?

    >

    >      >

    >

    >      > Any help would be appreciated,

    >

    >      >

    >

    >      > Thanks!

    >

    >      >

    >

    >      > -Justin

    >

    >      >

    >

    >      > project.xml:

    >

    >      >

    >

    >      > <source name="dbsnp-test" type="dbsnp-test">

    >

    >      >

    >

    >      > <property name="geneSource" value="Ensembl95"/>

    >

    >      >

    >

    >      > <property name="src.data.dir.includes" value="*.vcf"/>

    >

    >      >

    >

    >      > <property name="src.data.dir" location="/db/intermine/data/part_I"/>

    >

    >      >

    >

    >      > </source>

    >

    >      >

    >

    >      > [letourneaujj@intermineTest dbsnptest]$ cd

    >

    >      > /db/intermine/intermine/bio/sources/

    >

    >      >

    >

    >      > [letourneaujj@intermineTest dbsnptest]$

    > ./intermine-scripts/make_source

    >

    >      > dbsnp-test custom-file

    >

    >      >

    >

    >      > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB

    >

    >      >

    >

    >      >> Configure project :

    >

    >      >

    >

    >      > Using properties file:

    > '/home/letourneaujj/.intermine/dbsnptest.properties'

    >

    >      >

    >

    >      > cleaning

    >

    >      >

    >

    >      > deleting idresolver cache

    >

    >      >

    >

    >      > deleting datasources.xml

    >

    >      >

    >

    >      > deleting organisms.xml

    >

    >      >

    >

    >      > deleting publications.xml

    >

    >      >

    >

    >      > deleted log files

    >

    >      >

    >

    >      > FAILURE: Build failed with an exception.

    >

    >      >

    >

    >      > * What went wrong:

    >

    >      >

    >

    >      > Execution failed for task ':dbmodel:generateKeys'.

    >

    >      >

    >

    >      >> Could not resolve all files for configuration

    > ':dbmodel:mergeSource'.

    >

    >      >

    >

    >      >     > Could not find any matches for

    >

    >      > org.intermine:bio-source-dbsnp-test:4+ as no versions of

    >

    >      > org.intermine:bio-source-dbsnp-test are available.

    >

    >      >

    >

    >      >       Searched in the following locations:

    >

    >      >

    >

    >      >

    >

    >      >

    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/maven-metadata.xml

    >

    >      >

    >

    >      >

    >

    >      >

    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/

    >

    >      >

    >

    >      >

    >

    >      >

    > https://jcenter.bintray.com/org/intermine/bio-source-testing/maven-metadata.xml

    >

    >      >

    >

    >      >          

    > https://jcenter.bintray.com/org/intermine/bio-source-testing/

    >

    >      >

    >

    >      >

    >

    >      >

    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/maven-metadata.xml

    >

    >      >

    >

    >      >

    >

    >      >

    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/

    >

    >      >

    >

    >      >       Required by:

    >

    >      >

    >

    >      >           project :dbmodel

    >

    >      >

    >

    >      > * Try:

    >

    >      >

    >

    >      > Run with --stacktrace option to get the stack trace. Run with

    > --info or

    >

    >      > --debug option to get more log output.

    >

    >      >

    >

    >      > * Get more help at https://help.gradle.org

    >

    >      >

    >

    >      > BUILD FAILED in 9s

    >

    >      >

    >

    >      > 10 actionable tasks: 8 executed, 2 up-to-date

    >

    >      >

    >

    


_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Custom Datasource Error

Le Tourneau, Justin J.
In reply to this post by Julie Sullivan-2

Hi Julie,

 

Not to inundate you with info, but here’s what I tried today:

 

Create a fresh install of everything:

 

$ git clone https://github.com/intermine/intermine.git intermine

$ intermine-scripts/make_mine dbsnptest

$ cd bio/sources/

$ /db/intermine/intermine/intermine-scripts/make_source dbsnp-variation custom-file

$ ./gradlew install

 

FAILURE: Build failed with an exception.

 

* What went wrong:

Execution failed for task ':bio-source-dbsnp-variation:copyBioGenomicModel'.

> Expected configuration ':bio-source-dbsnp-variation:bioModel' to contain exactly one file, however, it contains no files.

 

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

 

* Get more help at https://help.gradle.org

 

BUILD FAILED in 6s

25 actionable tasks: 12 executed, 13 up-to-date

 

 

It fails. If I then try to just build the database, it fails:

 

$ cd ../../dbsnptest/

$ ./gradlew clean builddb

 

> Configure project :

Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'

cleaning

deleting idresolver cache

deleting datasources.xml

deleting organisms.xml

deleting publications.xml

deleted log files

 

 

FAILURE: Build failed with an exception.

 

* What went wrong:

Execution failed for task ':dbmodel:generateKeys'.

> Could not resolve all files for configuration ':dbmodel:mergeSource'.

   > Could not find any matches for org.intermine:bio-source-dbsnp-variation:4.0.+ as no versions of org.intermine:bio-source-dbsnp-variation are available.

     Searched in the following locations:

         file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml

         file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-variation/

         https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml

         https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-variation/

         https://clojars.org/repo/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml

         https://clojars.org/repo/org/intermine/bio-source-dbsnp-variation/

         https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml

         https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/

         http://www.ebi.ac.uk/intact/maven/nexus/content/repositories/ebi-repo/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml

         http://www.ebi.ac.uk/intact/maven/nexus/content/repositories/ebi-repo/org/intermine/bio-source-dbsnp-variation/

     Required by:

         project :dbmodel

 

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

 

* Get more help at https://help.gradle.org

 

BUILD FAILED in 5s

9 actionable tasks: 7 executed, 2 up-to-date

 

 

and if I specify the version in the project.xml… it fails

 

$ ./gradlew clean builddb

 

> Configure project :

Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'

cleaning

deleting idresolver cache

deleting datasources.xml

deleting organisms.xml

deleting publications.xml

deleted log files

 

 

FAILURE: Build failed with an exception.

 

* What went wrong:

Execution failed for task ':dbmodel:generateKeys'.

> Could not resolve all files for configuration ':dbmodel:mergeSource'.

   > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.

     Required by:

         project :dbmodel

      > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.

         > Could not get resource 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'.

            > Could not GET 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'. Received status code 409 from server:

 

* Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

 

* Get more help at https://help.gradle.org

 

BUILD FAILED in 5s

9 actionable tasks: 7 executed, 2 up-to-date

 

 

I’m not able to get anything besides the biotest to work at all.

 

Thanks for any help,

 

-Justin

 

 

 

 

 

On 7/25/19, 11:08 AM, "Julie Sullivan" <[hidden email]> wrote:

 

    Hi Justin,

   

    No problem at all!

   

    HOW TO PUT YOUR SOURCE ON THE CLASSPATH

   

    The build assumes that your code is compiled, in a JAR and on the

    classpath. To do that, run this command:

   

                ~/bio/sources $ ./gradlew install

   

    You should then see your JAR in your maven directory. If you make any

    changes to your code, you will want to re-run that command -- to

    recompile and make another JAR.

   

    WHY MAVEN IS LOOKING IN THE REMOTE REPO

   

    We have Maven configured to look locally then to look in Maven central.

    We have to look both places for dependencies. Your source is local, but

    most of your dependencies (e.g. apache-commons) will be in the remote

    maven repo.

   

    Here is where we set the repos:

   

    https://github.com/intermine/flymine-bio-sources/blob/master/build.gradle#L37

   

             mavenLocal() <-- looks locally

             jcenter() <-- on failure, it will look in remote repo

   

    So it checks locally first, fails to find your JAR. Then it looks in the

    remote repo, and fails too. That's why you see that error.

   

    I hope that helps!

    Julie

   

    On 25/07/2019 16:52, Le Tourneau, Justin J. wrote:

    > Hey Julie,

    >

    > Thanks for the quick reply. I have the version set in both places:

    >

    > build.gradle

    >

    > subprojects {

    >

    >      group = 'org.intermine'

    >

    >      version = '4.0.1'

    >

    > project.xml

    >

    > <source name="dbsnp-test" type="dbsnp-test" version=”4.0.1”>

    >

    > There’s nothing being written to the maven directory:

    >

    > [letourneaujj@intermineTest dbsnptest]$ ll ~/.m2/repository/

    >

    > total 0

    >

    > and, for whatever reason, it’s still looking remotely for these

    > resources??  This is the most confusing thing to me. Is there some way

    > for your build system to look locally?

    >

    > Sorry for being a pain, but I’m really scratching my head here.

    > Following the directions verbatim from the documentation should work

    > without any configuration changes, no?

    >

    > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB

    >

    >> Configure project :

    >

    > Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'

    >

    > cleaning

    >

    > deleting idresolver cache

    >

    > deleting datasources.xml

    >

    > deleting organisms.xml

    >

    > deleting publications.xml

    >

    > deleted log files

    >

    > FAILURE: Build failed with an exception.

    >

    > * What went wrong:

    >

    > Execution failed for task ':dbmodel:generateKeys'.

    >

    >> Could not resolve all files for configuration ':dbmodel:mergeSource'.

    >

    >     > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.

    >

    >       Required by:

    >

    >           project :dbmodel

    >

    >        > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.

    >

    >           > Could not get resource

    > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test

    > -4.0.1.pom'.

    >

    >              > Could not GET

    > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test

    > -4.0.1.pom'. Received status code 409 from server:

    >

    > * Try:

    >

    > Run with --stacktrace option to get the stack trace. Run with --info or

    > --debug option to get more log output.

    >

    > * Get more help at https://help.gradle.org

    >

    > BUILD FAILED in 5s

    >

    > 10 actionable tasks: 7 executed, 3 up-to-date

    >

    > On 7/25/19, 2:16 AM, "Julie Sullivan" <[hidden email]> wrote:

    >

    >      Hi Justin!

    >

    >      Here are the docs on versioning your sources:

    >

    >      https://intermine.readthedocs.io/en/latest/database/data-sources/versions/

    >

    >      There is an "InterMine version" set, which for you is 4.0.1 (I

    > think?).

    >

    >      This is what the sources default to, and why the build is looking for

    >

    >      that version. You can either:

    >

    >      option A: Set your custom source to be version 4.0.1

    >

    >      option B: Set your custom source manually in project XML (e.g.

    >

    >      version="1.0.0")

    >

    >      Either will work.

    >

    >      1. version is set in the build gradle file, e.g.

    >

    >      https://github.com/intermine/humanmine-bio-sources/blob/master/build.gradle#L24

    >

    >      2. You can verify your JAR was created with the correct version by

    >

    >      looking in the maven directory:

    >

    >       > ~/.m2/repository/org/intermine/bio-source-hpo $ ls

    >

    >       > 4.0.0  maven-metadata-local.xml

    >

    >      What version is your dbsnp source set at now?

    >

    >      Julie

    >

    >      PS. I've CCed our public developers mailing list as others may have

    > this

    >

    >      same issue!

    >

    >      https://lists.intermine.org/pipermail/dev/

    >

    >      On 24/07/2019 16:52, Le Tourneau, Justin J. wrote:

    >

    >      > Hi Julie!

    >

    >      >

    >

    >      > Dr. Christine Elsike and Shamim from my lab gave me your email

    >

    >      > concerning Intermine issues. I hope it’s okay to email you directly.

    >

    >      >

    >

    >      > I’ve installed and setup the latest version of Intermine without any

    >

    >      > issue (I’m trying to start to move our instances to the latest

    > version).

    >

    >      > However, when I try to create a new datasource (a test for dbsnp

    > data in

    >

    >      > this case) I get the error below. This is just after creating my

    >

    >      > ‘dbsnp-test’ source and following the directions from your

    > documentation

    >

    >      >

    > (https://intermine.readthedocs.io/en/latest/database/data-sources/custom/).

    >

    >      > The documentation is a bit confusing and unclear on versioning,

    > but, I

    >

    >      > think, I do not need to worry as this is just a new test instance

    > of the

    >

    >      > parser? I can just let it default to 4+ ?

    >

    >      >

    >

    >      > It seems to be looking for the source in my home directory and then

    >

    >      > remotely, is there a way to point it to the bio/sources directory?

    >

    >      >

    >

    >      > Any help would be appreciated,

    >

    >      >

    >

    >      > Thanks!

    >

    >      >

    >

    >      > -Justin

    >

    >      >

    >

    >      > project.xml:

    >

    >      >

    >

    >      > <source name="dbsnp-test" type="dbsnp-test">

    >

    >      >

    >

    >      > <property name="geneSource" value="Ensembl95"/>

    >

    >      >

    >

    >      > <property name="src.data.dir.includes" value="*.vcf"/>

    >

    >      >

    >

    >      > <property name="src.data.dir" location="/db/intermine/data/part_I"/>

    >

    >      >

    >

    >      > </source>

    >

    >      >

    >

    >      > [letourneaujj@intermineTest dbsnptest]$ cd

    >

    >      > /db/intermine/intermine/bio/sources/

    >

    >      >

    >

    >      > [letourneaujj@intermineTest dbsnptest]$

    > ./intermine-scripts/make_source

    >

    >      > dbsnp-test custom-file

    >

    >      >

    >

    >      > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB

    >

    >      >

    >

    >      >> Configure project :

    >

    >      >

    >

    >      > Using properties file:

    > '/home/letourneaujj/.intermine/dbsnptest.properties'

    >

    >      >

    >

    >      > cleaning

    >

    >      >

    >

    >      > deleting idresolver cache

    >

    >      >

    >

    >      > deleting datasources.xml

    >

    >      >

    >

    >      > deleting organisms.xml

    >

    >      >

    >

    >      > deleting publications.xml

    >

    >      >

    >

    >      > deleted log files

    >

    >      >

    >

    >      > FAILURE: Build failed with an exception.

    >

    >      >

    >

    >      > * What went wrong:

    >

    >      >

    >

    >      > Execution failed for task ':dbmodel:generateKeys'.

    >

    >      >

    >

    >      >> Could not resolve all files for configuration

    > ':dbmodel:mergeSource'.

    >

    >      >

    >

    >      >     > Could not find any matches for

    >

    >      > org.intermine:bio-source-dbsnp-test:4+ as no versions of

    >

    >      > org.intermine:bio-source-dbsnp-test are available.

    >

    >      >

    >

    >      >       Searched in the following locations:

    >

    >      >

    >

    >      >

    >

    >      >

    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/maven-metadata.xml

    >

    >      >

    >

    >      >

    >

    >      >

    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/

    >

    >      >

    >

    >      >

    >

    >      >

    > https://jcenter.bintray.com/org/intermine/bio-source-testing/maven-metadata.xml

    >

    >      >

    >

    >      >          

    > https://jcenter.bintray.com/org/intermine/bio-source-testing/

    >

    >      >

    >

    >      >

    >

    >      >

    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/maven-metadata.xml

    >

    >      >

    >

    >      >

    >

    >      >

    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/

    >

    >      >

    >

    >      >       Required by:

    >

    >      >

    >

    >      >           project :dbmodel

    >

    >      >

    >

    >      > * Try:

    >

    >      >

    >

    >      > Run with --stacktrace option to get the stack trace. Run with

    > --info or

    >

    >      > --debug option to get more log output.

    >

    >      >

    >

    >      > * Get more help at https://help.gradle.org

    >

    >      >

    >

    >      > BUILD FAILED in 9s

    >

    >      >

    >

    >      > 10 actionable tasks: 8 executed, 2 up-to-date

    >

    >      >

    >

    


_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Custom Datasource Error

Julie Sullivan-2
Hi Justin!

No worries, email as much as you need! :) FYI we are also on discord at
chat.intermine.org if that helps!

1. revert your changes to your gradle file. Your gradle file is correct.
2. Update the version of your data source. I detailed how to do that in
my last email. (just change version in your build.gradle to be "4.0.0")

Try that?
Julie

On 26/07/2019 21:13, Le Tourneau, Justin J. wrote:

> Hi Julie,
>
> Not to inundate you with info, but here’s what I tried today:
>
> Create a fresh install of everything:
>
> $ git clone https://github.com/intermine/intermine.git intermine
>
> $ intermine-scripts/make_mine dbsnptest
>
> $ cd bio/sources/
>
> $ /db/intermine/intermine/intermine-scripts/make_source dbsnp-variation
> custom-file
>
> $ ./gradlew install
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
>
> Execution failed for task ':bio-source-dbsnp-variation:copyBioGenomicModel'.
>
>> Expected configuration ':bio-source-dbsnp-variation:bioModel' to contain exactly one file, however, it contains no files.
>
> * Try:
>
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output.
>
> * Get more help at https://help.gradle.org
>
> BUILD FAILED in 6s
>
> 25 actionable tasks: 12 executed, 13 up-to-date
>
> It fails. If I then try to just build the database, it fails:
>
> $ cd ../../dbsnptest/
>
> $ ./gradlew clean builddb
>
>> Configure project :
>
> Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
>
> cleaning
>
> deleting idresolver cache
>
> deleting datasources.xml
>
> deleting organisms.xml
>
> deleting publications.xml
>
> deleted log files
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
>
> Execution failed for task ':dbmodel:generateKeys'.
>
>> Could not resolve all files for configuration ':dbmodel:mergeSource'.
>
>     > Could not find any matches for
> org.intermine:bio-source-dbsnp-variation:4.0.+ as no versions of
> org.intermine:bio-source-dbsnp-variation are available.
>
>       Searched in the following locations:
>
>          
> file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>
>          
> file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-variation/
>
>          
> https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>
>          
> https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-variation/
>
>          
> https://clojars.org/repo/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>
>           https://clojars.org/repo/org/intermine/bio-source-dbsnp-variation/
>
>          
> https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>
>          
> https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/
>
>          
> http://www.ebi.ac.uk/intact/maven/nexus/content/repositories/ebi-repo/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>
>          
> http://www.ebi.ac.uk/intact/maven/nexus/content/repositories/ebi-repo/org/intermine/bio-source-dbsnp-variation/
>
>       Required by:
>
>           project :dbmodel
>
> * Try:
>
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output.
>
> * Get more help at https://help.gradle.org
>
> BUILD FAILED in 5s
>
> 9 actionable tasks: 7 executed, 2 up-to-date
>
> and if I specify the version in the project.xml… it fails
>
> $ ./gradlew clean builddb
>
>> Configure project :
>
> Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
>
> cleaning
>
> deleting idresolver cache
>
> deleting datasources.xml
>
> deleting organisms.xml
>
> deleting publications.xml
>
> deleted log files
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
>
> Execution failed for task ':dbmodel:generateKeys'.
>
>> Could not resolve all files for configuration ':dbmodel:mergeSource'.
>
>     > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
>
>       Required by:
>
>           project :dbmodel
>
>        > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
>
>           > Could not get resource
> 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'.
>
>              > Could not GET
> 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'.
> Received status code 409 from server:
>
> * Try:
>
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output.
>
> * Get more help at https://help.gradle.org
>
> BUILD FAILED in 5s
>
> 9 actionable tasks: 7 executed, 2 up-to-date
>
> I’m not able to get anything besides the biotest to work at all.
>
> Thanks for any help,
>
> -Justin
>
> On 7/25/19, 11:08 AM, "Julie Sullivan" <[hidden email]> wrote:
>
>      Hi Justin,
>
>      No problem at all!
>
>      HOW TO PUT YOUR SOURCE ON THE CLASSPATH
>
>      The build assumes that your code is compiled, in a JAR and on the
>
>      classpath. To do that, run this command:
>
>                  ~/bio/sources $ ./gradlew install
>
>      You should then see your JAR in your maven directory. If you make any
>
>      changes to your code, you will want to re-run that command -- to
>
>      recompile and make another JAR.
>
>      WHY MAVEN IS LOOKING IN THE REMOTE REPO
>
>      We have Maven configured to look locally then to look in Maven
> central.
>
>      We have to look both places for dependencies. Your source is local,
> but
>
>      most of your dependencies (e.g. apache-commons) will be in the remote
>
>      maven repo.
>
>      Here is where we set the repos:
>
>      https://github.com/intermine/flymine-bio-sources/blob/master/build.gradle#L37
>
>               mavenLocal() <-- looks locally
>
>               jcenter() <-- on failure, it will look in remote repo
>
>      So it checks locally first, fails to find your JAR. Then it looks
> in the
>
>      remote repo, and fails too. That's why you see that error.
>
>      I hope that helps!
>
>      Julie
>
>      On 25/07/2019 16:52, Le Tourneau, Justin J. wrote:
>
>      > Hey Julie,
>
>      >
>
>      > Thanks for the quick reply. I have the version set in both places:
>
>      >
>
>      > build.gradle
>
>      >
>
>      > subprojects {
>
>      >
>
>      >      group = 'org.intermine'
>
>      >
>
>      >      version = '4.0.1'
>
>      >
>
>      > project.xml
>
>      >
>
>      > <source name="dbsnp-test" type="dbsnp-test" version=”4.0.1”>
>
>      >
>
>      > There’s nothing being written to the maven directory:
>
>      >
>
>      > [letourneaujj@intermineTest dbsnptest]$ ll ~/.m2/repository/
>
>      >
>
>      > total 0
>
>      >
>
>      > and, for whatever reason, it’s still looking remotely for these
>
>      > resources??  This is the most confusing thing to me. Is there
> some way
>
>      > for your build system to look locally?
>
>      >
>
>      > Sorry for being a pain, but I’m really scratching my head here.
>
>      > Following the directions verbatim from the documentation should work
>
>      > without any configuration changes, no?
>
>      >
>
>      > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB
>
>      >
>
>      >> Configure project :
>
>      >
>
>      > Using properties file:
> '/home/letourneaujj/.intermine/dbsnptest.properties'
>
>      >
>
>      > cleaning
>
>      >
>
>      > deleting idresolver cache
>
>      >
>
>      > deleting datasources.xml
>
>      >
>
>      > deleting organisms.xml
>
>      >
>
>      > deleting publications.xml
>
>      >
>
>      > deleted log files
>
>      >
>
>      > FAILURE: Build failed with an exception.
>
>      >
>
>      > * What went wrong:
>
>      >
>
>      > Execution failed for task ':dbmodel:generateKeys'.
>
>      >
>
>      >> Could not resolve all files for configuration
> ':dbmodel:mergeSource'.
>
>      >
>
>      >     > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.
>
>      >
>
>      >       Required by:
>
>      >
>
>      >           project :dbmodel
>
>      >
>
>      >        > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.
>
>      >
>
>      >           > Could not get resource
>
>      >
> 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test 
>
>
>      > -4.0.1.pom'.
>
>      >
>
>      >              > Could not GET
>
>      >
> 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test 
>
>
>      > -4.0.1.pom'. Received status code 409 from server:
>
>      >
>
>      > * Try:
>
>      >
>
>      > Run with --stacktrace option to get the stack trace. Run with
> --info or
>
>      > --debug option to get more log output.
>
>      >
>
>      > * Get more help at https://help.gradle.org
>
>      >
>
>      > BUILD FAILED in 5s
>
>      >
>
>      > 10 actionable tasks: 7 executed, 3 up-to-date
>
>      >
>
>      > On 7/25/19, 2:16 AM, "Julie Sullivan" <[hidden email]> wrote:
>
>      >
>
>      >      Hi Justin!
>
>      >
>
>      >      Here are the docs on versioning your sources:
>
>      >
>
>      >      
> https://intermine.readthedocs.io/en/latest/database/data-sources/versions/
>
>      >
>
>      >      There is an "InterMine version" set, which for you is 4.0.1 (I
>
>      > think?).
>
>      >
>
>      >      This is what the sources default to, and why the build is
> looking for
>
>      >
>
>      >      that version. You can either:
>
>      >
>
>      >      option A: Set your custom source to be version 4.0.1
>
>      >
>
>      >      option B: Set your custom source manually in project XML (e.g.
>
>      >
>
>      >      version="1.0.0")
>
>      >
>
>      >      Either will work.
>
>      >
>
>      >      1. version is set in the build gradle file, e.g.
>
>      >
>
>      >      
> https://github.com/intermine/humanmine-bio-sources/blob/master/build.gradle#L24
>
>      >
>
>      >      2. You can verify your JAR was created with the correct
> version by
>
>      >
>
>      >      looking in the maven directory:
>
>      >
>
>      >       > ~/.m2/repository/org/intermine/bio-source-hpo $ ls
>
>      >
>
>      >       > 4.0.0  maven-metadata-local.xml
>
>      >
>
>      >      What version is your dbsnp source set at now?
>
>      >
>
>      >      Julie
>
>      >
>
>      >      PS. I've CCed our public developers mailing list as others
> may have
>
>      > this
>
>      >
>
>      >      same issue!
>
>      >
>
>      >      https://lists.intermine.org/pipermail/dev/
>
>      >
>
>      >      On 24/07/2019 16:52, Le Tourneau, Justin J. wrote:
>
>      >
>
>      >      > Hi Julie!
>
>      >
>
>      >      >
>
>      >
>
>      >      > Dr. Christine Elsike and Shamim from my lab gave me your email
>
>      >
>
>      >      > concerning Intermine issues. I hope it’s okay to email you
> directly.
>
>      >
>
>      >      >
>
>      >
>
>      >      > I’ve installed and setup the latest version of Intermine
> without any
>
>      >
>
>      >      > issue (I’m trying to start to move our instances to the
> latest
>
>      > version).
>
>      >
>
>      >      > However, when I try to create a new datasource (a test for
> dbsnp
>
>      > data in
>
>      >
>
>      >      > this case) I get the error below. This is just after
> creating my
>
>      >
>
>      >      > ‘dbsnp-test’ source and following the directions from your
>
>      > documentation
>
>      >
>
>      >      >
>
>      >
> (https://intermine.readthedocs.io/en/latest/database/data-sources/custom/).
>
>      >
>
>      >      > The documentation is a bit confusing and unclear on
> versioning,
>
>      > but, I
>
>      >
>
>      >      > think, I do not need to worry as this is just a new test
> instance
>
>      > of the
>
>      >
>
>      >      > parser? I can just let it default to 4+ ?
>
>      >
>
>      >      >
>
>      >
>
>      >      > It seems to be looking for the source in my home directory
> and then
>
>      >
>
>      >      > remotely, is there a way to point it to the bio/sources
> directory?
>
>      >
>
>      >      >
>
>      >
>
>      >      > Any help would be appreciated,
>
>      >
>
>      >      >
>
>      >
>
>      >      > Thanks!
>
>      >
>
>      >      >
>
>      >
>
>      >      > -Justin
>
>      >
>
>      >      >
>
>      >
>
>      >      > project.xml:
>
>      >
>
>      >      >
>
>      >
>
>      >      > <source name="dbsnp-test" type="dbsnp-test">
>
>      >
>
>      >      >
>
>      >
>
>      >      > <property name="geneSource" value="Ensembl95"/>
>
>      >
>
>      >      >
>
>      >
>
>      >      > <property name="src.data.dir.includes" value="*.vcf"/>
>
>      >
>
>      >      >
>
>      >
>
>      >      > <property name="src.data.dir"
> location="/db/intermine/data/part_I"/>
>
>      >
>
>      >      >
>
>      >
>
>      >      > </source>
>
>      >
>
>      >      >
>
>      >
>
>      >      > [letourneaujj@intermineTest dbsnptest]$ cd
>
>      >
>
>      >      > /db/intermine/intermine/bio/sources/
>
>      >
>
>      >      >
>
>      >
>
>      >      > [letourneaujj@intermineTest dbsnptest]$
>
>      > ./intermine-scripts/make_source
>
>      >
>
>      >      > dbsnp-test custom-file
>
>      >
>
>      >      >
>
>      >
>
>      >      > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean
> buildDB
>
>      >
>
>      >      >
>
>      >
>
>      >      >> Configure project :
>
>      >
>
>      >      >
>
>      >
>
>      >      > Using properties file:
>
>      > '/home/letourneaujj/.intermine/dbsnptest.properties'
>
>      >
>
>      >      >
>
>      >
>
>      >      > cleaning
>
>      >
>
>      >      >
>
>      >
>
>      >      > deleting idresolver cache
>
>      >
>
>      >      >
>
>      >
>
>      >      > deleting datasources.xml
>
>      >
>
>      >      >
>
>      >
>
>      >      > deleting organisms.xml
>
>      >
>
>      >      >
>
>      >
>
>      >      > deleting publications.xml
>
>      >
>
>      >      >
>
>      >
>
>      >      > deleted log files
>
>      >
>
>      >      >
>
>      >
>
>      >      > FAILURE: Build failed with an exception.
>
>      >
>
>      >      >
>
>      >
>
>      >      > * What went wrong:
>
>      >
>
>      >      >
>
>      >
>
>      >      > Execution failed for task ':dbmodel:generateKeys'.
>
>      >
>
>      >      >
>
>      >
>
>      >      >> Could not resolve all files for configuration
>
>      > ':dbmodel:mergeSource'.
>
>      >
>
>      >      >
>
>      >
>
>      >      >     > Could not find any matches for
>
>      >
>
>      >      > org.intermine:bio-source-dbsnp-test:4+ as no versions of
>
>      >
>
>      >      > org.intermine:bio-source-dbsnp-test are available.
>
>      >
>
>      >      >
>
>      >
>
>      >      >       Searched in the following locations:
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
> file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/maven-metadata.xml
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
> file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
> https://jcenter.bintray.com/org/intermine/bio-source-testing/maven-metadata.xml
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      > https://jcenter.bintray.com/org/intermine/bio-source-testing/
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
> https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/maven-metadata.xml
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
>
>      >      >
>
>      >
> https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/
>
>      >
>
>      >      >
>
>      >
>
>      >      >       Required by:
>
>      >
>
>      >      >
>
>      >
>
>      >      >           project :dbmodel
>
>      >
>
>      >      >
>
>      >
>
>      >      > * Try:
>
>      >
>
>      >      >
>
>      >
>
>      >      > Run with --stacktrace option to get the stack trace. Run with
>
>      > --info or
>
>      >
>
>      >      > --debug option to get more log output.
>
>      >
>
>      >      >
>
>      >
>
>      >      > * Get more help at https://help.gradle.org
>
>      >
>
>      >      >
>
>      >
>
>      >      > BUILD FAILED in 9s
>
>      >
>
>      >      >
>
>      >
>
>      >      > 10 actionable tasks: 8 executed, 2 up-to-date
>
>      >
>
>      >      >
>
>      >
>
_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Custom Datasource Error

Le Tourneau, Justin J.
Thanks Julie,

I will join the discord under “support”


Gradle:

systemProp.imVersion=4.0.0
systemProp.bioVersion=4.0.0
systemProp.blueGenesVersion=0.9.9-SNAPSHOT

systemProp.javax.xml.stream.XMLOutputFactory=com.sun.xml.internal.stream.XMLOutputFactoryImpl


project.xml:

<project type="bio">
  <property name="target.model" value="genomic"/>
  <property name="common.os.prefix" value="common"/>
  <sources>

        <source name="Sscrofa11.1-dbsnp-variation-I" type="dbsnp-variation" version="4.0.0">
          <property name="dbsnp-variation.taxonId" value="9823"/>
          <property name="dbsnp-variation.assemblyVersion" value="Sscrofa11.1"/>
          <property name="dbsnp-variation.geneSource" value="Ensembl95"/>
          <property name="src.data.dir.includes" value="*.vcf"/>
          <property name="src.data.dir" location="/db/intermine/data/part_I"/>
        </source>
  </sources>

  <post-processing>
    <post-process name="create-references"/>
    <post-process name="transfer-sequences" dump="true"/>

    <!-- this runs the postprocessing steps in all sources -->
    <post-process name="do-sources"/>

    <post-process name="summarise-objectstore"/>
    <post-process name="create-autocomplete-index"/>
    <post-process name="create-search-index"/>
  </post-processing>


</project>

Failed:

[letourneaujj@intermineTest dbsnptest]$ ./gradlew clean builddb

> Configure project :
Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
cleaning
deleting idresolver cache
deleting datasources.xml
deleting organisms.xml
deleting publications.xml
deleted log files


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dbmodel:generateKeys'.
> Could not resolve all files for configuration ':dbmodel:mergeSource'.
   > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
     Required by:
         project :dbmodel
      > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
         > Could not get resource 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'.
            > Could not GET 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'. Received status code 409 from server:

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

* Get more help at https://help.gradle.org

BUILD FAILED in 5s
9 actionable tasks: 7 executed, 2 up-to-date




On 7/29/19, 1:46 AM, "Julie Sullivan" <[hidden email]> wrote:

    Hi Justin!
   
    No worries, email as much as you need! :) FYI we are also on discord at
    chat.intermine.org if that helps!
   
    1. revert your changes to your gradle file. Your gradle file is correct.
    2. Update the version of your data source. I detailed how to do that in
    my last email. (just change version in your build.gradle to be "4.0.0")
   
    Try that?
    Julie
   
    On 26/07/2019 21:13, Le Tourneau, Justin J. wrote:
    > Hi Julie,
    >
    > Not to inundate you with info, but here’s what I tried today:
    >
    > Create a fresh install of everything:
    >
    > $ git clone https://github.com/intermine/intermine.git intermine
    >
    > $ intermine-scripts/make_mine dbsnptest
    >
    > $ cd bio/sources/
    >
    > $ /db/intermine/intermine/intermine-scripts/make_source dbsnp-variation
    > custom-file
    >
    > $ ./gradlew install
    >
    > FAILURE: Build failed with an exception.
    >
    > * What went wrong:
    >
    > Execution failed for task ':bio-source-dbsnp-variation:copyBioGenomicModel'.
    >
    >> Expected configuration ':bio-source-dbsnp-variation:bioModel' to contain exactly one file, however, it contains no files.
    >
    > * Try:
    >
    > Run with --stacktrace option to get the stack trace. Run with --info or
    > --debug option to get more log output.
    >
    > * Get more help at https://help.gradle.org
    >
    > BUILD FAILED in 6s
    >
    > 25 actionable tasks: 12 executed, 13 up-to-date
    >
    > It fails. If I then try to just build the database, it fails:
    >
    > $ cd ../../dbsnptest/
    >
    > $ ./gradlew clean builddb
    >
    >> Configure project :
    >
    > Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
    >
    > cleaning
    >
    > deleting idresolver cache
    >
    > deleting datasources.xml
    >
    > deleting organisms.xml
    >
    > deleting publications.xml
    >
    > deleted log files
    >
    > FAILURE: Build failed with an exception.
    >
    > * What went wrong:
    >
    > Execution failed for task ':dbmodel:generateKeys'.
    >
    >> Could not resolve all files for configuration ':dbmodel:mergeSource'.
    >
    >     > Could not find any matches for
    > org.intermine:bio-source-dbsnp-variation:4.0.+ as no versions of
    > org.intermine:bio-source-dbsnp-variation are available.
    >
    >       Searched in the following locations:
    >
    >          
    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
    >
    >          
    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-variation/
    >
    >          
    > https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
    >
    >          
    > https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-variation/
    >
    >          
    > https://clojars.org/repo/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
    >
    >           https://clojars.org/repo/org/intermine/bio-source-dbsnp-variation/
    >
    >          
    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
    >
    >          
    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/
    >
    >          
    > http://www.ebi.ac.uk/intact/maven/nexus/content/repositories/ebi-repo/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
    >
    >          
    > http://www.ebi.ac.uk/intact/maven/nexus/content/repositories/ebi-repo/org/intermine/bio-source-dbsnp-variation/
    >
    >       Required by:
    >
    >           project :dbmodel
    >
    > * Try:
    >
    > Run with --stacktrace option to get the stack trace. Run with --info or
    > --debug option to get more log output.
    >
    > * Get more help at https://help.gradle.org
    >
    > BUILD FAILED in 5s
    >
    > 9 actionable tasks: 7 executed, 2 up-to-date
    >
    > and if I specify the version in the project.xml… it fails
    >
    > $ ./gradlew clean builddb
    >
    >> Configure project :
    >
    > Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
    >
    > cleaning
    >
    > deleting idresolver cache
    >
    > deleting datasources.xml
    >
    > deleting organisms.xml
    >
    > deleting publications.xml
    >
    > deleted log files
    >
    > FAILURE: Build failed with an exception.
    >
    > * What went wrong:
    >
    > Execution failed for task ':dbmodel:generateKeys'.
    >
    >> Could not resolve all files for configuration ':dbmodel:mergeSource'.
    >
    >     > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
    >
    >       Required by:
    >
    >           project :dbmodel
    >
    >        > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
    >
    >           > Could not get resource
    > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'.
    >
    >              > Could not GET
    > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'.
    > Received status code 409 from server:
    >
    > * Try:
    >
    > Run with --stacktrace option to get the stack trace. Run with --info or
    > --debug option to get more log output.
    >
    > * Get more help at https://help.gradle.org
    >
    > BUILD FAILED in 5s
    >
    > 9 actionable tasks: 7 executed, 2 up-to-date
    >
    > I’m not able to get anything besides the biotest to work at all.
    >
    > Thanks for any help,
    >
    > -Justin
    >
    > On 7/25/19, 11:08 AM, "Julie Sullivan" <[hidden email]> wrote:
    >
    >      Hi Justin,
    >
    >      No problem at all!
    >
    >      HOW TO PUT YOUR SOURCE ON THE CLASSPATH
    >
    >      The build assumes that your code is compiled, in a JAR and on the
    >
    >      classpath. To do that, run this command:
    >
    >                  ~/bio/sources $ ./gradlew install
    >
    >      You should then see your JAR in your maven directory. If you make any
    >
    >      changes to your code, you will want to re-run that command -- to
    >
    >      recompile and make another JAR.
    >
    >      WHY MAVEN IS LOOKING IN THE REMOTE REPO
    >
    >      We have Maven configured to look locally then to look in Maven
    > central.
    >
    >      We have to look both places for dependencies. Your source is local,
    > but
    >
    >      most of your dependencies (e.g. apache-commons) will be in the remote
    >
    >      maven repo.
    >
    >      Here is where we set the repos:
    >
    >      https://github.com/intermine/flymine-bio-sources/blob/master/build.gradle#L37
    >
    >               mavenLocal() <-- looks locally
    >
    >               jcenter() <-- on failure, it will look in remote repo
    >
    >      So it checks locally first, fails to find your JAR. Then it looks
    > in the
    >
    >      remote repo, and fails too. That's why you see that error.
    >
    >      I hope that helps!
    >
    >      Julie
    >
    >      On 25/07/2019 16:52, Le Tourneau, Justin J. wrote:
    >
    >      > Hey Julie,
    >
    >      >
    >
    >      > Thanks for the quick reply. I have the version set in both places:
    >
    >      >
    >
    >      > build.gradle
    >
    >      >
    >
    >      > subprojects {
    >
    >      >
    >
    >      >      group = 'org.intermine'
    >
    >      >
    >
    >      >      version = '4.0.1'
    >
    >      >
    >
    >      > project.xml
    >
    >      >
    >
    >      > <source name="dbsnp-test" type="dbsnp-test" version=”4.0.1”>
    >
    >      >
    >
    >      > There’s nothing being written to the maven directory:
    >
    >      >
    >
    >      > [letourneaujj@intermineTest dbsnptest]$ ll ~/.m2/repository/
    >
    >      >
    >
    >      > total 0
    >
    >      >
    >
    >      > and, for whatever reason, it’s still looking remotely for these
    >
    >      > resources??  This is the most confusing thing to me. Is there
    > some way
    >
    >      > for your build system to look locally?
    >
    >      >
    >
    >      > Sorry for being a pain, but I’m really scratching my head here.
    >
    >      > Following the directions verbatim from the documentation should work
    >
    >      > without any configuration changes, no?
    >
    >      >
    >
    >      > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB
    >
    >      >
    >
    >      >> Configure project :
    >
    >      >
    >
    >      > Using properties file:
    > '/home/letourneaujj/.intermine/dbsnptest.properties'
    >
    >      >
    >
    >      > cleaning
    >
    >      >
    >
    >      > deleting idresolver cache
    >
    >      >
    >
    >      > deleting datasources.xml
    >
    >      >
    >
    >      > deleting organisms.xml
    >
    >      >
    >
    >      > deleting publications.xml
    >
    >      >
    >
    >      > deleted log files
    >
    >      >
    >
    >      > FAILURE: Build failed with an exception.
    >
    >      >
    >
    >      > * What went wrong:
    >
    >      >
    >
    >      > Execution failed for task ':dbmodel:generateKeys'.
    >
    >      >
    >
    >      >> Could not resolve all files for configuration
    > ':dbmodel:mergeSource'.
    >
    >      >
    >
    >      >     > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.
    >
    >      >
    >
    >      >       Required by:
    >
    >      >
    >
    >      >           project :dbmodel
    >
    >      >
    >
    >      >        > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.
    >
    >      >
    >
    >      >           > Could not get resource
    >
    >      >
    > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test 
    >
    >
    >      > -4.0.1.pom'.
    >
    >      >
    >
    >      >              > Could not GET
    >
    >      >
    > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test 
    >
    >
    >      > -4.0.1.pom'. Received status code 409 from server:
    >
    >      >
    >
    >      > * Try:
    >
    >      >
    >
    >      > Run with --stacktrace option to get the stack trace. Run with
    > --info or
    >
    >      > --debug option to get more log output.
    >
    >      >
    >
    >      > * Get more help at https://help.gradle.org
    >
    >      >
    >
    >      > BUILD FAILED in 5s
    >
    >      >
    >
    >      > 10 actionable tasks: 7 executed, 3 up-to-date
    >
    >      >
    >
    >      > On 7/25/19, 2:16 AM, "Julie Sullivan" <[hidden email]> wrote:
    >
    >      >
    >
    >      >      Hi Justin!
    >
    >      >
    >
    >      >      Here are the docs on versioning your sources:
    >
    >      >
    >
    >      >      
    > https://intermine.readthedocs.io/en/latest/database/data-sources/versions/
    >
    >      >
    >
    >      >      There is an "InterMine version" set, which for you is 4.0.1 (I
    >
    >      > think?).
    >
    >      >
    >
    >      >      This is what the sources default to, and why the build is
    > looking for
    >
    >      >
    >
    >      >      that version. You can either:
    >
    >      >
    >
    >      >      option A: Set your custom source to be version 4.0.1
    >
    >      >
    >
    >      >      option B: Set your custom source manually in project XML (e.g.
    >
    >      >
    >
    >      >      version="1.0.0")
    >
    >      >
    >
    >      >      Either will work.
    >
    >      >
    >
    >      >      1. version is set in the build gradle file, e.g.
    >
    >      >
    >
    >      >      
    > https://github.com/intermine/humanmine-bio-sources/blob/master/build.gradle#L24
    >
    >      >
    >
    >      >      2. You can verify your JAR was created with the correct
    > version by
    >
    >      >
    >
    >      >      looking in the maven directory:
    >
    >      >
    >
    >      >       > ~/.m2/repository/org/intermine/bio-source-hpo $ ls
    >
    >      >
    >
    >      >       > 4.0.0  maven-metadata-local.xml
    >
    >      >
    >
    >      >      What version is your dbsnp source set at now?
    >
    >      >
    >
    >      >      Julie
    >
    >      >
    >
    >      >      PS. I've CCed our public developers mailing list as others
    > may have
    >
    >      > this
    >
    >      >
    >
    >      >      same issue!
    >
    >      >
    >
    >      >      https://lists.intermine.org/pipermail/dev/
    >
    >      >
    >
    >      >      On 24/07/2019 16:52, Le Tourneau, Justin J. wrote:
    >
    >      >
    >
    >      >      > Hi Julie!
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > Dr. Christine Elsike and Shamim from my lab gave me your email
    >
    >      >
    >
    >      >      > concerning Intermine issues. I hope it’s okay to email you
    > directly.
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > I’ve installed and setup the latest version of Intermine
    > without any
    >
    >      >
    >
    >      >      > issue (I’m trying to start to move our instances to the
    > latest
    >
    >      > version).
    >
    >      >
    >
    >      >      > However, when I try to create a new datasource (a test for
    > dbsnp
    >
    >      > data in
    >
    >      >
    >
    >      >      > this case) I get the error below. This is just after
    > creating my
    >
    >      >
    >
    >      >      > ‘dbsnp-test’ source and following the directions from your
    >
    >      > documentation
    >
    >      >
    >
    >      >      >
    >
    >      >
    > (https://intermine.readthedocs.io/en/latest/database/data-sources/custom/).
    >
    >      >
    >
    >      >      > The documentation is a bit confusing and unclear on
    > versioning,
    >
    >      > but, I
    >
    >      >
    >
    >      >      > think, I do not need to worry as this is just a new test
    > instance
    >
    >      > of the
    >
    >      >
    >
    >      >      > parser? I can just let it default to 4+ ?
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > It seems to be looking for the source in my home directory
    > and then
    >
    >      >
    >
    >      >      > remotely, is there a way to point it to the bio/sources
    > directory?
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > Any help would be appreciated,
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > Thanks!
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > -Justin
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > project.xml:
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > <source name="dbsnp-test" type="dbsnp-test">
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > <property name="geneSource" value="Ensembl95"/>
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > <property name="src.data.dir.includes" value="*.vcf"/>
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > <property name="src.data.dir"
    > location="/db/intermine/data/part_I"/>
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > </source>
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > [letourneaujj@intermineTest dbsnptest]$ cd
    >
    >      >
    >
    >      >      > /db/intermine/intermine/bio/sources/
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > [letourneaujj@intermineTest dbsnptest]$
    >
    >      > ./intermine-scripts/make_source
    >
    >      >
    >
    >      >      > dbsnp-test custom-file
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean
    > buildDB
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >> Configure project :
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > Using properties file:
    >
    >      > '/home/letourneaujj/.intermine/dbsnptest.properties'
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > cleaning
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > deleting idresolver cache
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > deleting datasources.xml
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > deleting organisms.xml
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > deleting publications.xml
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > deleted log files
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > FAILURE: Build failed with an exception.
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > * What went wrong:
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > Execution failed for task ':dbmodel:generateKeys'.
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >> Could not resolve all files for configuration
    >
    >      > ':dbmodel:mergeSource'.
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >     > Could not find any matches for
    >
    >      >
    >
    >      >      > org.intermine:bio-source-dbsnp-test:4+ as no versions of
    >
    >      >
    >
    >      >      > org.intermine:bio-source-dbsnp-test are available.
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >       Searched in the following locations:
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/maven-metadata.xml
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    > https://jcenter.bintray.com/org/intermine/bio-source-testing/maven-metadata.xml
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      > https://jcenter.bintray.com/org/intermine/bio-source-testing/
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/maven-metadata.xml
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >
    >
    >      >
    > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >       Required by:
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      >           project :dbmodel
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > * Try:
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > Run with --stacktrace option to get the stack trace. Run with
    >
    >      > --info or
    >
    >      >
    >
    >      >      > --debug option to get more log output.
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > * Get more help at https://help.gradle.org
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > BUILD FAILED in 9s
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
    >      >      > 10 actionable tasks: 8 executed, 2 up-to-date
    >
    >      >
    >
    >      >      >
    >
    >      >
    >
   

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Custom Datasource Error

Julie Sullivan-2
You should update your version to be 4.0.0. It's still looking for
version 4.0:

    > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.

Which version is your JAR?

You can see which version was created by looking at your JAR in your
maven repository under ~/.m2/repository. I am guessing it's version 2.0.0.

On 30/07/2019 17:15, Le Tourneau, Justin J. wrote:

> Thanks Julie,
>
> I will join the discord under “support”
>
>
> Gradle:
>
> systemProp.imVersion=4.0.0
> systemProp.bioVersion=4.0.0
> systemProp.blueGenesVersion=0.9.9-SNAPSHOT
>
> systemProp.javax.xml.stream.XMLOutputFactory=com.sun.xml.internal.stream.XMLOutputFactoryImpl
>
>
> project.xml:
>
> <project type="bio">
>    <property name="target.model" value="genomic"/>
>    <property name="common.os.prefix" value="common"/>
>    <sources>
>
>          <source name="Sscrofa11.1-dbsnp-variation-I" type="dbsnp-variation" version="4.0.0">
>            <property name="dbsnp-variation.taxonId" value="9823"/>
>            <property name="dbsnp-variation.assemblyVersion" value="Sscrofa11.1"/>
>            <property name="dbsnp-variation.geneSource" value="Ensembl95"/>
>            <property name="src.data.dir.includes" value="*.vcf"/>
>            <property name="src.data.dir" location="/db/intermine/data/part_I"/>
>          </source>
>    </sources>
>
>    <post-processing>
>      <post-process name="create-references"/>
>      <post-process name="transfer-sequences" dump="true"/>
>
>      <!-- this runs the postprocessing steps in all sources -->
>      <post-process name="do-sources"/>
>
>      <post-process name="summarise-objectstore"/>
>      <post-process name="create-autocomplete-index"/>
>      <post-process name="create-search-index"/>
>    </post-processing>
>
>
> </project>
>
> Failed:
>
> [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean builddb
>
>> Configure project :
> Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
> cleaning
> deleting idresolver cache
> deleting datasources.xml
> deleting organisms.xml
> deleting publications.xml
> deleted log files
>
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
> Execution failed for task ':dbmodel:generateKeys'.
>> Could not resolve all files for configuration ':dbmodel:mergeSource'.
>     > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
>       Required by:
>           project :dbmodel
>        > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
>           > Could not get resource 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'.
>              > Could not GET 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'. Received status code 409 from server:
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
>
> * Get more help at https://help.gradle.org
>
> BUILD FAILED in 5s
> 9 actionable tasks: 7 executed, 2 up-to-date
>
>
>
>
> On 7/29/19, 1:46 AM, "Julie Sullivan" <[hidden email]> wrote:
>
>      Hi Justin!
>      
>      No worries, email as much as you need! :) FYI we are also on discord at
>      chat.intermine.org if that helps!
>      
>      1. revert your changes to your gradle file. Your gradle file is correct.
>      2. Update the version of your data source. I detailed how to do that in
>      my last email. (just change version in your build.gradle to be "4.0.0")
>      
>      Try that?
>      Julie
>      
>      On 26/07/2019 21:13, Le Tourneau, Justin J. wrote:
>      > Hi Julie,
>      >
>      > Not to inundate you with info, but here’s what I tried today:
>      >
>      > Create a fresh install of everything:
>      >
>      > $ git clone https://github.com/intermine/intermine.git intermine
>      >
>      > $ intermine-scripts/make_mine dbsnptest
>      >
>      > $ cd bio/sources/
>      >
>      > $ /db/intermine/intermine/intermine-scripts/make_source dbsnp-variation
>      > custom-file
>      >
>      > $ ./gradlew install
>      >
>      > FAILURE: Build failed with an exception.
>      >
>      > * What went wrong:
>      >
>      > Execution failed for task ':bio-source-dbsnp-variation:copyBioGenomicModel'.
>      >
>      >> Expected configuration ':bio-source-dbsnp-variation:bioModel' to contain exactly one file, however, it contains no files.
>      >
>      > * Try:
>      >
>      > Run with --stacktrace option to get the stack trace. Run with --info or
>      > --debug option to get more log output.
>      >
>      > * Get more help at https://help.gradle.org
>      >
>      > BUILD FAILED in 6s
>      >
>      > 25 actionable tasks: 12 executed, 13 up-to-date
>      >
>      > It fails. If I then try to just build the database, it fails:
>      >
>      > $ cd ../../dbsnptest/
>      >
>      > $ ./gradlew clean builddb
>      >
>      >> Configure project :
>      >
>      > Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
>      >
>      > cleaning
>      >
>      > deleting idresolver cache
>      >
>      > deleting datasources.xml
>      >
>      > deleting organisms.xml
>      >
>      > deleting publications.xml
>      >
>      > deleted log files
>      >
>      > FAILURE: Build failed with an exception.
>      >
>      > * What went wrong:
>      >
>      > Execution failed for task ':dbmodel:generateKeys'.
>      >
>      >> Could not resolve all files for configuration ':dbmodel:mergeSource'.
>      >
>      >     > Could not find any matches for
>      > org.intermine:bio-source-dbsnp-variation:4.0.+ as no versions of
>      > org.intermine:bio-source-dbsnp-variation are available.
>      >
>      >       Searched in the following locations:
>      >
>      >
>      > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>      >
>      >
>      > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-dbsnp-variation/
>      >
>      >
>      > https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>      >
>      >
>      > https://jcenter.bintray.com/org/intermine/bio-source-dbsnp-variation/
>      >
>      >
>      > https://clojars.org/repo/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>      >
>      >           https://clojars.org/repo/org/intermine/bio-source-dbsnp-variation/
>      >
>      >
>      > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>      >
>      >
>      > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/
>      >
>      >
>      > http://www.ebi.ac.uk/intact/maven/nexus/content/repositories/ebi-repo/org/intermine/bio-source-dbsnp-variation/maven-metadata.xml
>      >
>      >
>      > http://www.ebi.ac.uk/intact/maven/nexus/content/repositories/ebi-repo/org/intermine/bio-source-dbsnp-variation/
>      >
>      >       Required by:
>      >
>      >           project :dbmodel
>      >
>      > * Try:
>      >
>      > Run with --stacktrace option to get the stack trace. Run with --info or
>      > --debug option to get more log output.
>      >
>      > * Get more help at https://help.gradle.org
>      >
>      > BUILD FAILED in 5s
>      >
>      > 9 actionable tasks: 7 executed, 2 up-to-date
>      >
>      > and if I specify the version in the project.xml… it fails
>      >
>      > $ ./gradlew clean builddb
>      >
>      >> Configure project :
>      >
>      > Using properties file: '/home/letourneaujj/.intermine/dbsnptest.properties'
>      >
>      > cleaning
>      >
>      > deleting idresolver cache
>      >
>      > deleting datasources.xml
>      >
>      > deleting organisms.xml
>      >
>      > deleting publications.xml
>      >
>      > deleted log files
>      >
>      > FAILURE: Build failed with an exception.
>      >
>      > * What went wrong:
>      >
>      > Execution failed for task ':dbmodel:generateKeys'.
>      >
>      >> Could not resolve all files for configuration ':dbmodel:mergeSource'.
>      >
>      >     > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
>      >
>      >       Required by:
>      >
>      >           project :dbmodel
>      >
>      >        > Could not resolve org.intermine:bio-source-dbsnp-variation:4.0.0.
>      >
>      >           > Could not get resource
>      > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'.
>      >
>      >              > Could not GET
>      > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-dbsnp-variation/4.0.0/bio-source-dbsnp-variation-4.0.0.pom'.
>      > Received status code 409 from server:
>      >
>      > * Try:
>      >
>      > Run with --stacktrace option to get the stack trace. Run with --info or
>      > --debug option to get more log output.
>      >
>      > * Get more help at https://help.gradle.org
>      >
>      > BUILD FAILED in 5s
>      >
>      > 9 actionable tasks: 7 executed, 2 up-to-date
>      >
>      > I’m not able to get anything besides the biotest to work at all.
>      >
>      > Thanks for any help,
>      >
>      > -Justin
>      >
>      > On 7/25/19, 11:08 AM, "Julie Sullivan" <[hidden email]> wrote:
>      >
>      >      Hi Justin,
>      >
>      >      No problem at all!
>      >
>      >      HOW TO PUT YOUR SOURCE ON THE CLASSPATH
>      >
>      >      The build assumes that your code is compiled, in a JAR and on the
>      >
>      >      classpath. To do that, run this command:
>      >
>      >                  ~/bio/sources $ ./gradlew install
>      >
>      >      You should then see your JAR in your maven directory. If you make any
>      >
>      >      changes to your code, you will want to re-run that command -- to
>      >
>      >      recompile and make another JAR.
>      >
>      >      WHY MAVEN IS LOOKING IN THE REMOTE REPO
>      >
>      >      We have Maven configured to look locally then to look in Maven
>      > central.
>      >
>      >      We have to look both places for dependencies. Your source is local,
>      > but
>      >
>      >      most of your dependencies (e.g. apache-commons) will be in the remote
>      >
>      >      maven repo.
>      >
>      >      Here is where we set the repos:
>      >
>      >      https://github.com/intermine/flymine-bio-sources/blob/master/build.gradle#L37
>      >
>      >               mavenLocal() <-- looks locally
>      >
>      >               jcenter() <-- on failure, it will look in remote repo
>      >
>      >      So it checks locally first, fails to find your JAR. Then it looks
>      > in the
>      >
>      >      remote repo, and fails too. That's why you see that error.
>      >
>      >      I hope that helps!
>      >
>      >      Julie
>      >
>      >      On 25/07/2019 16:52, Le Tourneau, Justin J. wrote:
>      >
>      >      > Hey Julie,
>      >
>      >      >
>      >
>      >      > Thanks for the quick reply. I have the version set in both places:
>      >
>      >      >
>      >
>      >      > build.gradle
>      >
>      >      >
>      >
>      >      > subprojects {
>      >
>      >      >
>      >
>      >      >      group = 'org.intermine'
>      >
>      >      >
>      >
>      >      >      version = '4.0.1'
>      >
>      >      >
>      >
>      >      > project.xml
>      >
>      >      >
>      >
>      >      > <source name="dbsnp-test" type="dbsnp-test" version=”4.0.1”>
>      >
>      >      >
>      >
>      >      > There’s nothing being written to the maven directory:
>      >
>      >      >
>      >
>      >      > [letourneaujj@intermineTest dbsnptest]$ ll ~/.m2/repository/
>      >
>      >      >
>      >
>      >      > total 0
>      >
>      >      >
>      >
>      >      > and, for whatever reason, it’s still looking remotely for these
>      >
>      >      > resources??  This is the most confusing thing to me. Is there
>      > some way
>      >
>      >      > for your build system to look locally?
>      >
>      >      >
>      >
>      >      > Sorry for being a pain, but I’m really scratching my head here.
>      >
>      >      > Following the directions verbatim from the documentation should work
>      >
>      >      > without any configuration changes, no?
>      >
>      >      >
>      >
>      >      > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean buildDB
>      >
>      >      >
>      >
>      >      >> Configure project :
>      >
>      >      >
>      >
>      >      > Using properties file:
>      > '/home/letourneaujj/.intermine/dbsnptest.properties'
>      >
>      >      >
>      >
>      >      > cleaning
>      >
>      >      >
>      >
>      >      > deleting idresolver cache
>      >
>      >      >
>      >
>      >      > deleting datasources.xml
>      >
>      >      >
>      >
>      >      > deleting organisms.xml
>      >
>      >      >
>      >
>      >      > deleting publications.xml
>      >
>      >      >
>      >
>      >      > deleted log files
>      >
>      >      >
>      >
>      >      > FAILURE: Build failed with an exception.
>      >
>      >      >
>      >
>      >      > * What went wrong:
>      >
>      >      >
>      >
>      >      > Execution failed for task ':dbmodel:generateKeys'.
>      >
>      >      >
>      >
>      >      >> Could not resolve all files for configuration
>      > ':dbmodel:mergeSource'.
>      >
>      >      >
>      >
>      >      >     > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.
>      >
>      >      >
>      >
>      >      >       Required by:
>      >
>      >      >
>      >
>      >      >           project :dbmodel
>      >
>      >      >
>      >
>      >      >        > Could not resolve org.intermine:bio-source-dbsnp-test:4.0.1.
>      >
>      >      >
>      >
>      >      >           > Could not get resource
>      >
>      >      >
>      > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test
>      >
>      >
>      >      > -4.0.1.pom'.
>      >
>      >      >
>      >
>      >      >              > Could not GET
>      >
>      >      >
>      > 'https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/4.0.1/bio-source-dbsnp-test
>      >
>      >
>      >      > -4.0.1.pom'. Received status code 409 from server:
>      >
>      >      >
>      >
>      >      > * Try:
>      >
>      >      >
>      >
>      >      > Run with --stacktrace option to get the stack trace. Run with
>      > --info or
>      >
>      >      > --debug option to get more log output.
>      >
>      >      >
>      >
>      >      > * Get more help at https://help.gradle.org
>      >
>      >      >
>      >
>      >      > BUILD FAILED in 5s
>      >
>      >      >
>      >
>      >      > 10 actionable tasks: 7 executed, 3 up-to-date
>      >
>      >      >
>      >
>      >      > On 7/25/19, 2:16 AM, "Julie Sullivan" <[hidden email]> wrote:
>      >
>      >      >
>      >
>      >      >      Hi Justin!
>      >
>      >      >
>      >
>      >      >      Here are the docs on versioning your sources:
>      >
>      >      >
>      >
>      >      >
>      > https://intermine.readthedocs.io/en/latest/database/data-sources/versions/
>      >
>      >      >
>      >
>      >      >      There is an "InterMine version" set, which for you is 4.0.1 (I
>      >
>      >      > think?).
>      >
>      >      >
>      >
>      >      >      This is what the sources default to, and why the build is
>      > looking for
>      >
>      >      >
>      >
>      >      >      that version. You can either:
>      >
>      >      >
>      >
>      >      >      option A: Set your custom source to be version 4.0.1
>      >
>      >      >
>      >
>      >      >      option B: Set your custom source manually in project XML (e.g.
>      >
>      >      >
>      >
>      >      >      version="1.0.0")
>      >
>      >      >
>      >
>      >      >      Either will work.
>      >
>      >      >
>      >
>      >      >      1. version is set in the build gradle file, e.g.
>      >
>      >      >
>      >
>      >      >
>      > https://github.com/intermine/humanmine-bio-sources/blob/master/build.gradle#L24
>      >
>      >      >
>      >
>      >      >      2. You can verify your JAR was created with the correct
>      > version by
>      >
>      >      >
>      >
>      >      >      looking in the maven directory:
>      >
>      >      >
>      >
>      >      >       > ~/.m2/repository/org/intermine/bio-source-hpo $ ls
>      >
>      >      >
>      >
>      >      >       > 4.0.0  maven-metadata-local.xml
>      >
>      >      >
>      >
>      >      >      What version is your dbsnp source set at now?
>      >
>      >      >
>      >
>      >      >      Julie
>      >
>      >      >
>      >
>      >      >      PS. I've CCed our public developers mailing list as others
>      > may have
>      >
>      >      > this
>      >
>      >      >
>      >
>      >      >      same issue!
>      >
>      >      >
>      >
>      >      >      https://lists.intermine.org/pipermail/dev/
>      >
>      >      >
>      >
>      >      >      On 24/07/2019 16:52, Le Tourneau, Justin J. wrote:
>      >
>      >      >
>      >
>      >      >      > Hi Julie!
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > Dr. Christine Elsike and Shamim from my lab gave me your email
>      >
>      >      >
>      >
>      >      >      > concerning Intermine issues. I hope it’s okay to email you
>      > directly.
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > I’ve installed and setup the latest version of Intermine
>      > without any
>      >
>      >      >
>      >
>      >      >      > issue (I’m trying to start to move our instances to the
>      > latest
>      >
>      >      > version).
>      >
>      >      >
>      >
>      >      >      > However, when I try to create a new datasource (a test for
>      > dbsnp
>      >
>      >      > data in
>      >
>      >      >
>      >
>      >      >      > this case) I get the error below. This is just after
>      > creating my
>      >
>      >      >
>      >
>      >      >      > ‘dbsnp-test’ source and following the directions from your
>      >
>      >      > documentation
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      > (https://intermine.readthedocs.io/en/latest/database/data-sources/custom/).
>      >
>      >      >
>      >
>      >      >      > The documentation is a bit confusing and unclear on
>      > versioning,
>      >
>      >      > but, I
>      >
>      >      >
>      >
>      >      >      > think, I do not need to worry as this is just a new test
>      > instance
>      >
>      >      > of the
>      >
>      >      >
>      >
>      >      >      > parser? I can just let it default to 4+ ?
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > It seems to be looking for the source in my home directory
>      > and then
>      >
>      >      >
>      >
>      >      >      > remotely, is there a way to point it to the bio/sources
>      > directory?
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > Any help would be appreciated,
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > Thanks!
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > -Justin
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > project.xml:
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > <source name="dbsnp-test" type="dbsnp-test">
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > <property name="geneSource" value="Ensembl95"/>
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > <property name="src.data.dir.includes" value="*.vcf"/>
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > <property name="src.data.dir"
>      > location="/db/intermine/data/part_I"/>
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > </source>
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > [letourneaujj@intermineTest dbsnptest]$ cd
>      >
>      >      >
>      >
>      >      >      > /db/intermine/intermine/bio/sources/
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > [letourneaujj@intermineTest dbsnptest]$
>      >
>      >      > ./intermine-scripts/make_source
>      >
>      >      >
>      >
>      >      >      > dbsnp-test custom-file
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > [letourneaujj@intermineTest dbsnptest]$ ./gradlew clean
>      > buildDB
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >> Configure project :
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > Using properties file:
>      >
>      >      > '/home/letourneaujj/.intermine/dbsnptest.properties'
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > cleaning
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > deleting idresolver cache
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > deleting datasources.xml
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > deleting organisms.xml
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > deleting publications.xml
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > deleted log files
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > FAILURE: Build failed with an exception.
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > * What went wrong:
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > Execution failed for task ':dbmodel:generateKeys'.
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >> Could not resolve all files for configuration
>      >
>      >      > ':dbmodel:mergeSource'.
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >     > Could not find any matches for
>      >
>      >      >
>      >
>      >      >      > org.intermine:bio-source-dbsnp-test:4+ as no versions of
>      >
>      >      >
>      >
>      >      >      > org.intermine:bio-source-dbsnp-test are available.
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >       Searched in the following locations:
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/maven-metadata.xml
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      > file:/home/letourneaujj/.m2/repository/org/intermine/bio-source-testing/
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      > https://jcenter.bintray.com/org/intermine/bio-source-testing/maven-metadata.xml
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      > https://jcenter.bintray.com/org/intermine/bio-source-testing/
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/maven-metadata.xml
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      > https://oss.jfrog.org/artifactory/oss-snapshot-local/org/intermine/bio-source-testing/
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >       Required by:
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      >           project :dbmodel
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > * Try:
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > Run with --stacktrace option to get the stack trace. Run with
>      >
>      >      > --info or
>      >
>      >      >
>      >
>      >      >      > --debug option to get more log output.
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > * Get more help at https://help.gradle.org
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > BUILD FAILED in 9s
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      >      >      > 10 actionable tasks: 8 executed, 2 up-to-date
>      >
>      >      >
>      >
>      >      >      >
>      >
>      >      >
>      >
>      
>
_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev