differential expression

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

differential expression

Sofia Robb-4
Can some one send me a sample of how you load differential expression data from different experiments into intermine? A small sample of your intermine xml or what ever file type you use and your datasource information would be very very helpful. modencode has some very nice pages with differential expression data that I would love to replicate. http://intermine.modencode.org/release-33/bagDetails.do?scope=global&bagName=example

Thank you very much,
Sofia

_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: differential expression

Sergio Contrino
dear sofia,
you can see the code related to the expression data in this repository
https://github.com/modENCODE-DCC/modmine
more specifically

https://github.com/modENCODE-DCC/modmine/tree/dev/bio/sources/modmine/fly-expression-score

an example of the data used is below, and you can wget the full file here

http://www.flymine.org/download/modmine/misc/Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt

the display is done with an old canvasXpress, you could probably use the
updated version or switch to something else.
i hope this helps
thank you
sergio



Gene_FBgn       ANNOTATION_SYMBOL       NAME    SYMBOL  Chromosome
CME_L1  Sg4     ML-DmD11        ML-DmD20-c2     ML-DmD20-c5     Kc167
GM2     S2-DRSC S2R+    S1      1182-4H ML-DmD16-c3     ML-DmD32
ML-DmD17-c3     ML-DmD8 CME W1 cl.8+    ML-DmD9 ML-DmBG1-c1     ML-DmD21
        ML-DmD4-c1      ML-DmBG3-c2     S3      CME W2  mbn2
ML-DmBG2-c2     emb_0-2h        emb_2-4h        emb_4-6h        emb_6-8h
        emb_8-10h       emb_10-12h      emb_12-14h      emb_14-16h
emb_16-18h      emb_18-20h      emb_20-22h      emb_22-24h
L1_larvae       L2_larvae       L3_larvae_12hr  L3_larvae_dark_blue
L3_larvae_light_blue    L3_larvae_clear WPP     WPP_12hr        WPP_24hr
        WPP_2days       WPP_3days       WPP_4days       Adult_F_Ecl_1day
        Adult_M_Ecl_1day        Adult_F_Ecl_5day        Adult_M_Ecl_5day
        Adult_F_Ecl_30day       Adult_M_Ecl_30day
FBgn0000003     CR32864 RNA 7SL 7SLRNA  chr3R   2865    766     1081
2554    6007    28853   15965   19318   40438   12516   11627   6284
21483   16001   86      28398   9037    19780   18895   6302    0
11805   11449   8065    7327    3169    19549   18903   0       742
2041    2104    10655   2660    11762   16171   9668    12804   13167
10390   6637    4434    3346    0       4852    0       2184    7606
8059    8887    6918    2544    5340    5663    1474
FBgn0000008     CG6741  arc     a       chr2R   207     131     349
246     370     292     103     179     128     200     343     239
211     164     519     336     110     397     303     210     180
57      140     149     204     1069    461     284     280     384
586     689     591     262     348     286     310     201     286
247     181     227     415     556     487     757     1027    630
366     320     269     430     219     359     273
FBgn0000014     CG10325 abdominal A     abd-A   chr3R   176     91
90      143     114     89      43      87      144     92      37
54      106     115     32      55      76      271     101     112
64      223     57      26      88      55      183     647     984
1018    779     636     627     435     322     243     331     290
311     334     536     625     692     888     460     503     346
212     156     169     148     58      81      120     86
FBgn0000015     CG11648 Abdominal B     Abd-B   chr3R   645     1172
101     95      86      99      62      67      111     45      117
76      57      89      117     120     69      104     79      28
60      1709    41      48      54      101     136     574     474
566     281     348     239     189     154     152     144     157
81      152     89      106     143     96      63      104     97
86      59      75      74      39      73      48      78
FBgn0000017     CG4032  Abl tyrosine kinase     Abl     chr3L   2076
1216    1315    1402    2413    1184    2013    1583    2253    1758
1802    1252    3079    1169    2281    1487    3399    1036    4081
1864    2052    1026    1791    1308    899     2711    2307    1093
958     1010    1564    1334    948     687     706     862     614
591     483     474     740     518     830     854     699     662
990     431     407     618     336     768     513     637     511
FBgn0000018     CG6093  abnormal oocyte abo     chr2L   166     264
244     313     266     213     208     305     364     269     233
163     175     158     184     191     230     330     235     267
178     172     236     490     198     627     11      88      68
29      30      33      15      27      33      47      14      15
27      37      56      88      51      58      29      37      47
15      22      87      60      235     65      132     40
FBgn0000022     CG3796  achaete ac      chrX    1       0       6
0       8       27      5       27      31      0       352     49
25      0       20      0       0       59      30      3       40
4       0       7       0       69      465     1018    482     302
40      0       10      0       14      45      19      13      1
28      47      52      46      61      85      69      6       0
3       18      9       31      20      18      10
FBgn0000024     CG17907 Acetylcholine esterase  Ace     chr3R   2471
714     2119    2049    753     106     547     1404    84      96
223     394     1984    196     836     321     149     32      431
1530    801     2478    57      56      56      287     51      63
67      34      103     147     408     365     436     561     550
551     528     603     156     221     265     349     456     316
500     670     530     401     416     136     377     123     435
FBgn0000028     CG9151  abnormal chemosensory jump 6    acj6    chrX
84      111     171     146     146     187     83      122     253
96      92      115     132     89      109     332     196     144
131     76      169     98      184     112     204     106     135
302     139     63      54      132     164     150     96      140
76      78      77      144     197     65      55      86      60
169     110     126     130     89      91      92      81      135     155


On 05/11/15 20:00, Sofia Robb wrote:

> Can some one send me a sample of how you load differential expression
> data from different experiments into intermine? A small sample of your
> intermine xml or what ever file type you use and your datasource
> information would be very very helpful. modencode has some very nice
> pages with differential expression data that I would love to replicate.
> http://intermine.modencode.org/release-33/bagDetails.do?scope=global&bagName=example
>
> Thank you very much,
> Sofia
>
>
> _______________________________________________
> dev mailing list
> [hidden email]
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>

--
sergio contrino                  InterMine, University of Cambridge
https://sergiocontrino.github.io           http://www.intermine.org

_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: differential expression

Sofia Robb-4

So this file (Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt) can be directly loaded with the fly-expression-source source? Also, what is a data converter? Do I have to create one for my organism specific data?

Thank you very much for all the info!
Sofia

On Fri, Nov 6, 2015 at 2:49 AM, sergio contrino <[hidden email]> wrote:
dear sofia,
you can see the code related to the expression data in this repository
https://github.com/modENCODE-DCC/modmine
more specifically

https://github.com/modENCODE-DCC/modmine/tree/dev/bio/sources/modmine/fly-expression-score

an example of the data used is below, and you can wget the full file here

http://www.flymine.org/download/modmine/misc/Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt

the display is done with an old canvasXpress, you could probably use the updated version or switch to something else.
i hope this helps
thank you
sergio



Gene_FBgn       ANNOTATION_SYMBOL       NAME    SYMBOL  Chromosome CME_L1  Sg4     ML-DmD11        ML-DmD20-c2     ML-DmD20-c5     Kc167 GM2     S2-DRSC S2R+    S1      1182-4H ML-DmD16-c3     ML-DmD32 ML-DmD17-c3     ML-DmD8 CME W1 cl.8+    ML-DmD9 ML-DmBG1-c1     ML-DmD21        ML-DmD4-c1      ML-DmBG3-c2     S3      CME W2  mbn2 ML-DmBG2-c2     emb_0-2h        emb_2-4h        emb_4-6h        emb_6-8h        emb_8-10h       emb_10-12h      emb_12-14h      emb_14-16h emb_16-18h      emb_18-20h      emb_20-22h      emb_22-24h L1_larvae       L2_larvae       L3_larvae_12hr  L3_larvae_dark_blue L3_larvae_light_blue    L3_larvae_clear WPP     WPP_12hr        WPP_24hr        WPP_2days       WPP_3days       WPP_4days       Adult_F_Ecl_1day        Adult_M_Ecl_1day        Adult_F_Ecl_5day        Adult_M_Ecl_5day        Adult_F_Ecl_30day       Adult_M_Ecl_30day
FBgn0000003     CR32864 RNA 7SL 7SLRNA  chr3R   2865    766     1081 2554    6007    28853   15965   19318   40438   12516   11627   6284 21483   16001   86      28398   9037    19780   18895   6302    0 11805   11449   8065    7327    3169    19549   18903   0       742 2041    2104    10655   2660    11762   16171   9668    12804   13167 10390   6637    4434    3346    0       4852    0       2184    7606 8059    8887    6918    2544    5340    5663    1474
FBgn0000008     CG6741  arc     a       chr2R   207     131     349 246     370     292     103     179     128     200     343     239 211     164     519     336     110     397     303     210     180 57      140     149     204     1069    461     284     280     384 586     689     591     262     348     286     310     201     286 247     181     227     415     556     487     757     1027    630 366     320     269     430     219     359     273
FBgn0000014     CG10325 abdominal A     abd-A   chr3R   176     91 90      143     114     89      43      87      144     92      37 54      106     115     32      55      76      271     101     112 64      223     57      26      88      55      183     647     984 1018    779     636     627     435     322     243     331     290 311     334     536     625     692     888     460     503     346 212     156     169     148     58      81      120     86
FBgn0000015     CG11648 Abdominal B     Abd-B   chr3R   645     1172 101     95      86      99      62      67      111     45      117 76      57      89      117     120     69      104     79      28 60      1709    41      48      54      101     136     574     474 566     281     348     239     189     154     152     144     157 81      152     89      106     143     96      63      104     97 86      59      75      74      39      73      48      78
FBgn0000017     CG4032  Abl tyrosine kinase     Abl     chr3L   2076 1216    1315    1402    2413    1184    2013    1583    2253    1758 1802    1252    3079    1169    2281    1487    3399    1036    4081 1864    2052    1026    1791    1308    899     2711    2307    1093 958     1010    1564    1334    948     687     706     862     614 591     483     474     740     518     830     854     699     662 990     431     407     618     336     768     513     637     511
FBgn0000018     CG6093  abnormal oocyte abo     chr2L   166     264 244     313     266     213     208     305     364     269     233 163     175     158     184     191     230     330     235     267 178     172     236     490     198     627     11      88      68 29      30      33      15      27      33      47      14      15 27      37      56      88      51      58      29      37      47 15      22      87      60      235     65      132     40
FBgn0000022     CG3796  achaete ac      chrX    1       0       6 0       8       27      5       27      31      0       352     49 25      0       20      0       0       59      30      3       40 4       0       7       0       69      465     1018    482     302 40      0       10      0       14      45      19      13      1 28      47      52      46      61      85      69      6       0 3       18      9       31      20      18      10
FBgn0000024     CG17907 Acetylcholine esterase  Ace     chr3R   2471 714     2119    2049    753     106     547     1404    84      96 223     394     1984    196     836     321     149     32      431 1530    801     2478    57      56      56      287     51      63 67      34      103     147     408     365     436     561     550 551     528     603     156     221     265     349     456     316 500     670     530     401     416     136     377     123     435
FBgn0000028     CG9151  abnormal chemosensory jump 6    acj6    chrX 84      111     171     146     146     187     83      122     253 96      92      115     132     89      109     332     196     144 131     76      169     98      184     112     204     106     135 302     139     63      54      132     164     150     96      140 76      78      77      144     197     65      55      86      60 169     110     126     130     89      91      92      81      135     155



On 05/11/15 20:00, Sofia Robb wrote:
Can some one send me a sample of how you load differential expression
data from different experiments into intermine? A small sample of your
intermine xml or what ever file type you use and your datasource
information would be very very helpful. modencode has some very nice
pages with differential expression data that I would love to replicate.
http://intermine.modencode.org/release-33/bagDetails.do?scope=global&bagName=example

Thank you very much,
Sofia


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev


--
sergio contrino                  InterMine, University of Cambridge
https://sergiocontrino.github.io           http://www.intermine.org


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: differential expression

Sofia Robb-4
In reply to this post by Sergio Contrino
Ok, I tried to load the above file but failed. 

I created a directory for a file that had just the first 100 lines of Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt (/home/SGC/smr/data/intermine/simrmine/flyexpression/flyexpression.txt)

I added this to my project.xml:
    <source name="flybase-expression" type="flybase-expression">
      <property name="src.data.dir" location="/home/SGC/smr/data/intermine/simrmine/flyexpression/"/>
    </source>

In my integrate directory I ran
ant -Dsource=flybase-expression -v from 


and i get this error:

-retrieve-tgt-from-custom-file:
[integrate] Processing file: /home/SGC/smr/data/intermine/simrmine/flyexpression/flyexpression.txt
[integrate] Creating id resolver from database and caching id resolver to file: build/idresolver.cache
     [null] Exiting /var/lib/pgsql/other_data/intermine/bio/sources/fly/flybase-expression/build.xml.

BUILD FAILED
/var/lib/pgsql/other_data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
/var/lib/pgsql/other_data/intermine/imbuild/source.xml:145: java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileReader.<init>(FileReader.java:72)
at org.intermine.bio.dataconversion.FlybaseExpressionConverter.process(FlybaseExpressionConverter.java:101)
at org.intermine.task.FileConverterTask.execute(FileConverterTask.java:116)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:435)
at org.apache.tools.ant.Target.performTasks(Target.java:456)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:440)
at org.intermine.task.Integrate.performAction(Integrate.java:223)
at org.intermine.task.Integrate.performAction(Integrate.java:135)
at org.intermine.task.Integrate.execute(Integrate.java:127)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:435)
at org.apache.tools.ant.Target.performTasks(Target.java:456)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
at org.apache.tools.ant.Main.runBuild(Main.java:851)
at org.apache.tools.ant.Main.startAnt(Main.java:235)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)


On Fri, Nov 6, 2015 at 2:49 AM, sergio contrino <[hidden email]> wrote:
dear sofia,
you can see the code related to the expression data in this repository
https://github.com/modENCODE-DCC/modmine
more specifically

https://github.com/modENCODE-DCC/modmine/tree/dev/bio/sources/modmine/fly-expression-score

an example of the data used is below, and you can wget the full file here

http://www.flymine.org/download/modmine/misc/Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt

the display is done with an old canvasXpress, you could probably use the updated version or switch to something else.
i hope this helps
thank you
sergio



Gene_FBgn       ANNOTATION_SYMBOL       NAME    SYMBOL  Chromosome CME_L1  Sg4     ML-DmD11        ML-DmD20-c2     ML-DmD20-c5     Kc167 GM2     S2-DRSC S2R+    S1      1182-4H ML-DmD16-c3     ML-DmD32 ML-DmD17-c3     ML-DmD8 CME W1 cl.8+    ML-DmD9 ML-DmBG1-c1     ML-DmD21        ML-DmD4-c1      ML-DmBG3-c2     S3      CME W2  mbn2 ML-DmBG2-c2     emb_0-2h        emb_2-4h        emb_4-6h        emb_6-8h        emb_8-10h       emb_10-12h      emb_12-14h      emb_14-16h emb_16-18h      emb_18-20h      emb_20-22h      emb_22-24h L1_larvae       L2_larvae       L3_larvae_12hr  L3_larvae_dark_blue L3_larvae_light_blue    L3_larvae_clear WPP     WPP_12hr        WPP_24hr        WPP_2days       WPP_3days       WPP_4days       Adult_F_Ecl_1day        Adult_M_Ecl_1day        Adult_F_Ecl_5day        Adult_M_Ecl_5day        Adult_F_Ecl_30day       Adult_M_Ecl_30day
FBgn0000003     CR32864 RNA 7SL 7SLRNA  chr3R   2865    766     1081 2554    6007    28853   15965   19318   40438   12516   11627   6284 21483   16001   86      28398   9037    19780   18895   6302    0 11805   11449   8065    7327    3169    19549   18903   0       742 2041    2104    10655   2660    11762   16171   9668    12804   13167 10390   6637    4434    3346    0       4852    0       2184    7606 8059    8887    6918    2544    5340    5663    1474
FBgn0000008     CG6741  arc     a       chr2R   207     131     349 246     370     292     103     179     128     200     343     239 211     164     519     336     110     397     303     210     180 57      140     149     204     1069    461     284     280     384 586     689     591     262     348     286     310     201     286 247     181     227     415     556     487     757     1027    630 366     320     269     430     219     359     273
FBgn0000014     CG10325 abdominal A     abd-A   chr3R   176     91 90      143     114     89      43      87      144     92      37 54      106     115     32      55      76      271     101     112 64      223     57      26      88      55      183     647     984 1018    779     636     627     435     322     243     331     290 311     334     536     625     692     888     460     503     346 212     156     169     148     58      81      120     86
FBgn0000015     CG11648 Abdominal B     Abd-B   chr3R   645     1172 101     95      86      99      62      67      111     45      117 76      57      89      117     120     69      104     79      28 60      1709    41      48      54      101     136     574     474 566     281     348     239     189     154     152     144     157 81      152     89      106     143     96      63      104     97 86      59      75      74      39      73      48      78
FBgn0000017     CG4032  Abl tyrosine kinase     Abl     chr3L   2076 1216    1315    1402    2413    1184    2013    1583    2253    1758 1802    1252    3079    1169    2281    1487    3399    1036    4081 1864    2052    1026    1791    1308    899     2711    2307    1093 958     1010    1564    1334    948     687     706     862     614 591     483     474     740     518     830     854     699     662 990     431     407     618     336     768     513     637     511
FBgn0000018     CG6093  abnormal oocyte abo     chr2L   166     264 244     313     266     213     208     305     364     269     233 163     175     158     184     191     230     330     235     267 178     172     236     490     198     627     11      88      68 29      30      33      15      27      33      47      14      15 27      37      56      88      51      58      29      37      47 15      22      87      60      235     65      132     40
FBgn0000022     CG3796  achaete ac      chrX    1       0       6 0       8       27      5       27      31      0       352     49 25      0       20      0       0       59      30      3       40 4       0       7       0       69      465     1018    482     302 40      0       10      0       14      45      19      13      1 28      47      52      46      61      85      69      6       0 3       18      9       31      20      18      10
FBgn0000024     CG17907 Acetylcholine esterase  Ace     chr3R   2471 714     2119    2049    753     106     547     1404    84      96 223     394     1984    196     836     321     149     32      431 1530    801     2478    57      56      56      287     51      63 67      34      103     147     408     365     436     561     550 551     528     603     156     221     265     349     456     316 500     670     530     401     416     136     377     123     435
FBgn0000028     CG9151  abnormal chemosensory jump 6    acj6    chrX 84      111     171     146     146     187     83      122     253 96      92      115     132     89      109     332     196     144 131     76      169     98      184     112     204     106     135 302     139     63      54      132     164     150     96      140 76      78      77      144     197     65      55      86      60 169     110     126     130     89      91      92      81      135     155



On 05/11/15 20:00, Sofia Robb wrote:
Can some one send me a sample of how you load differential expression
data from different experiments into intermine? A small sample of your
intermine xml or what ever file type you use and your datasource
information would be very very helpful. modencode has some very nice
pages with differential expression data that I would love to replicate.
http://intermine.modencode.org/release-33/bagDetails.do?scope=global&bagName=example

Thank you very much,
Sofia


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev


--
sergio contrino                  InterMine, University of Cambridge
https://sergiocontrino.github.io           http://www.intermine.org


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: differential expression

Sofia Robb-4
I switched my data source to the modmine fly-expression-score. I think this is the one I should use. And I get much more informative errors. From these errors, I think I need the modmine/fly-expression-score_additions.xml file. The one I have is has no classes defined.  I think it would be good to get the fly data to load, then I can modify my data and FlyExpressionScoreConverter.java to fit my data.

Caused by: java.lang.IllegalArgumentException: class "org.intermine.model.bio.Submission" is not in the Model


BUILD FAILED
/var/lib/pgsql/other_data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
/var/lib/pgsql/other_data/intermine/imbuild/source.xml:145: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.intermine.task.FileConverterTask.execute(FileConverterTask.java:97)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:435)
at org.apache.tools.ant.Target.performTasks(Target.java:456)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:440)
at org.intermine.task.Integrate.performAction(Integrate.java:223)
at org.intermine.task.Integrate.performAction(Integrate.java:135)
at org.intermine.task.Integrate.execute(Integrate.java:127)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:435)
at org.apache.tools.ant.Target.performTasks(Target.java:456)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
at org.apache.tools.ant.Main.runBuild(Main.java:851)
at org.apache.tools.ant.Main.startAnt(Main.java:235)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
Caused by: java.lang.IllegalArgumentException: class "org.intermine.model.bio.Submission" is not in the Model
at org.intermine.xml.full.Item.getClassDescriptorByName(Item.java:546)
at org.intermine.xml.full.Item.setClassName(Item.java:128)
at org.intermine.xml.full.ItemFactory.makeItem(ItemFactory.java:114)
at org.intermine.dataconversion.DataConverter.createItem(DataConverter.java:104)
at org.intermine.bio.dataconversion.FlyExpressionScoreConverter.createSubmissionItem(FlyExpressionScoreConverter.java:414)
at org.intermine.bio.dataconversion.FlyExpressionScoreConverter.<init>(FlyExpressionScoreConverter.java:65)
... 36 more

On Fri, Nov 6, 2015 at 12:55 PM, Sofia Robb <[hidden email]> wrote:
Ok, I tried to load the above file but failed. 

I created a directory for a file that had just the first 100 lines of Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt (/home/SGC/smr/data/intermine/simrmine/flyexpression/flyexpression.txt)

I added this to my project.xml:
    <source name="flybase-expression" type="flybase-expression">
      <property name="src.data.dir" location="/home/SGC/smr/data/intermine/simrmine/flyexpression/"/>
    </source>

In my integrate directory I ran
ant -Dsource=flybase-expression -v from 


and i get this error:

-retrieve-tgt-from-custom-file:
[integrate] Processing file: /home/SGC/smr/data/intermine/simrmine/flyexpression/flyexpression.txt
[integrate] Creating id resolver from database and caching id resolver to file: build/idresolver.cache
     [null] Exiting /var/lib/pgsql/other_data/intermine/bio/sources/fly/flybase-expression/build.xml.

BUILD FAILED
/var/lib/pgsql/other_data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
/var/lib/pgsql/other_data/intermine/imbuild/source.xml:145: java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileReader.<init>(FileReader.java:72)
at org.intermine.bio.dataconversion.FlybaseExpressionConverter.process(FlybaseExpressionConverter.java:101)
at org.intermine.task.FileConverterTask.execute(FileConverterTask.java:116)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:435)
at org.apache.tools.ant.Target.performTasks(Target.java:456)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:440)
at org.intermine.task.Integrate.performAction(Integrate.java:223)
at org.intermine.task.Integrate.performAction(Integrate.java:135)
at org.intermine.task.Integrate.execute(Integrate.java:127)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:435)
at org.apache.tools.ant.Target.performTasks(Target.java:456)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
at org.apache.tools.ant.Main.runBuild(Main.java:851)
at org.apache.tools.ant.Main.startAnt(Main.java:235)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)


On Fri, Nov 6, 2015 at 2:49 AM, sergio contrino <[hidden email]> wrote:
dear sofia,
you can see the code related to the expression data in this repository
https://github.com/modENCODE-DCC/modmine
more specifically

https://github.com/modENCODE-DCC/modmine/tree/dev/bio/sources/modmine/fly-expression-score

an example of the data used is below, and you can wget the full file here

http://www.flymine.org/download/modmine/misc/Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt

the display is done with an old canvasXpress, you could probably use the updated version or switch to something else.
i hope this helps
thank you
sergio



Gene_FBgn       ANNOTATION_SYMBOL       NAME    SYMBOL  Chromosome CME_L1  Sg4     ML-DmD11        ML-DmD20-c2     ML-DmD20-c5     Kc167 GM2     S2-DRSC S2R+    S1      1182-4H ML-DmD16-c3     ML-DmD32 ML-DmD17-c3     ML-DmD8 CME W1 cl.8+    ML-DmD9 ML-DmBG1-c1     ML-DmD21        ML-DmD4-c1      ML-DmBG3-c2     S3      CME W2  mbn2 ML-DmBG2-c2     emb_0-2h        emb_2-4h        emb_4-6h        emb_6-8h        emb_8-10h       emb_10-12h      emb_12-14h      emb_14-16h emb_16-18h      emb_18-20h      emb_20-22h      emb_22-24h L1_larvae       L2_larvae       L3_larvae_12hr  L3_larvae_dark_blue L3_larvae_light_blue    L3_larvae_clear WPP     WPP_12hr        WPP_24hr        WPP_2days       WPP_3days       WPP_4days       Adult_F_Ecl_1day        Adult_M_Ecl_1day        Adult_F_Ecl_5day        Adult_M_Ecl_5day        Adult_F_Ecl_30day       Adult_M_Ecl_30day
FBgn0000003     CR32864 RNA 7SL 7SLRNA  chr3R   2865    766     1081 2554    6007    28853   15965   19318   40438   12516   11627   6284 21483   16001   86      28398   9037    19780   18895   6302    0 11805   11449   8065    7327    3169    19549   18903   0       742 2041    2104    10655   2660    11762   16171   9668    12804   13167 10390   6637    4434    3346    0       4852    0       2184    7606 8059    8887    6918    2544    5340    5663    1474
FBgn0000008     CG6741  arc     a       chr2R   207     131     349 246     370     292     103     179     128     200     343     239 211     164     519     336     110     397     303     210     180 57      140     149     204     1069    461     284     280     384 586     689     591     262     348     286     310     201     286 247     181     227     415     556     487     757     1027    630 366     320     269     430     219     359     273
FBgn0000014     CG10325 abdominal A     abd-A   chr3R   176     91 90      143     114     89      43      87      144     92      37 54      106     115     32      55      76      271     101     112 64      223     57      26      88      55      183     647     984 1018    779     636     627     435     322     243     331     290 311     334     536     625     692     888     460     503     346 212     156     169     148     58      81      120     86
FBgn0000015     CG11648 Abdominal B     Abd-B   chr3R   645     1172 101     95      86      99      62      67      111     45      117 76      57      89      117     120     69      104     79      28 60      1709    41      48      54      101     136     574     474 566     281     348     239     189     154     152     144     157 81      152     89      106     143     96      63      104     97 86      59      75      74      39      73      48      78
FBgn0000017     CG4032  Abl tyrosine kinase     Abl     chr3L   2076 1216    1315    1402    2413    1184    2013    1583    2253    1758 1802    1252    3079    1169    2281    1487    3399    1036    4081 1864    2052    1026    1791    1308    899     2711    2307    1093 958     1010    1564    1334    948     687     706     862     614 591     483     474     740     518     830     854     699     662 990     431     407     618     336     768     513     637     511
FBgn0000018     CG6093  abnormal oocyte abo     chr2L   166     264 244     313     266     213     208     305     364     269     233 163     175     158     184     191     230     330     235     267 178     172     236     490     198     627     11      88      68 29      30      33      15      27      33      47      14      15 27      37      56      88      51      58      29      37      47 15      22      87      60      235     65      132     40
FBgn0000022     CG3796  achaete ac      chrX    1       0       6 0       8       27      5       27      31      0       352     49 25      0       20      0       0       59      30      3       40 4       0       7       0       69      465     1018    482     302 40      0       10      0       14      45      19      13      1 28      47      52      46      61      85      69      6       0 3       18      9       31      20      18      10
FBgn0000024     CG17907 Acetylcholine esterase  Ace     chr3R   2471 714     2119    2049    753     106     547     1404    84      96 223     394     1984    196     836     321     149     32      431 1530    801     2478    57      56      56      287     51      63 67      34      103     147     408     365     436     561     550 551     528     603     156     221     265     349     456     316 500     670     530     401     416     136     377     123     435
FBgn0000028     CG9151  abnormal chemosensory jump 6    acj6    chrX 84      111     171     146     146     187     83      122     253 96      92      115     132     89      109     332     196     144 131     76      169     98      184     112     204     106     135 302     139     63      54      132     164     150     96      140 76      78      77      144     197     65      55      86      60 169     110     126     130     89      91      92      81      135     155



On 05/11/15 20:00, Sofia Robb wrote:
Can some one send me a sample of how you load differential expression
data from different experiments into intermine? A small sample of your
intermine xml or what ever file type you use and your datasource
information would be very very helpful. modencode has some very nice
pages with differential expression data that I would love to replicate.
http://intermine.modencode.org/release-33/bagDetails.do?scope=global&bagName=example

Thank you very much,
Sofia


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev


--
sergio contrino                  InterMine, University of Cambridge
https://sergiocontrino.github.io           http://www.intermine.org



_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: differential expression

Sergio Contrino
dear sofia,
apologies for the slow response. most of the model modification for
modencode are in the file ~(in the usual repository)

https://github.com/modENCODE-DCC/modmine/blob/dev/modmine/dbmodel/resources/modencode-metadata_additions.xml

so yes, you need to adapt the loader to your model. for example you
probably don't need to deal with submissions.

please let me know if this yours does not supersede your previous mail
and i'll have a look at it today.

thank you!
sergio



On 09/11/15 18:48, Sofia Robb wrote:

> I switched my data source to the modmine fly-expression-score. I think
> this is the one I should use. And I get much more informative errors.
>  From these errors, I think I need the
> modmine/fly-expression-score_additions.xml file. The one I have is has
> no classes defined.  I think it would be good to get the fly data to
> load, then I can modify my data and FlyExpressionScoreConverter.java to
> fit my data.
>
> Caused by: java.lang.IllegalArgumentException: class
> "org.intermine.model.bio.Submission" is not in the Model
>
>
> BUILD FAILED
> /var/lib/pgsql/other_data/intermine/imbuild/integrate.xml:54: The
> following error occurred while executing this line:
> /var/lib/pgsql/other_data/intermine/imbuild/source.xml:145:
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.intermine.task.FileConverterTask.execute(FileConverterTask.java:97)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:435)
> at org.apache.tools.ant.Target.performTasks(Target.java:456)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
> at
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
> at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
> at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:440)
> at org.intermine.task.Integrate.performAction(Integrate.java:223)
> at org.intermine.task.Integrate.performAction(Integrate.java:135)
> at org.intermine.task.Integrate.execute(Integrate.java:127)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:435)
> at org.apache.tools.ant.Target.performTasks(Target.java:456)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
> at
> org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
> at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
> at org.apache.tools.ant.Main.runBuild(Main.java:851)
> at org.apache.tools.ant.Main.startAnt(Main.java:235)
> at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
> at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
> Caused by: java.lang.IllegalArgumentException: class
> "org.intermine.model.bio.Submission" is not in the Model
> at org.intermine.xml.full.Item.getClassDescriptorByName(Item.java:546)
> at org.intermine.xml.full.Item.setClassName(Item.java:128)
> at org.intermine.xml.full.ItemFactory.makeItem(ItemFactory.java:114)
> at
> org.intermine.dataconversion.DataConverter.createItem(DataConverter.java:104)
> at
> org.intermine.bio.dataconversion.FlyExpressionScoreConverter.createSubmissionItem(FlyExpressionScoreConverter.java:414)
> at
> org.intermine.bio.dataconversion.FlyExpressionScoreConverter.<init>(FlyExpressionScoreConverter.java:65)
> ... 36 more
>
> On Fri, Nov 6, 2015 at 12:55 PM, Sofia Robb <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Ok, I tried to load the above file but failed.
>
>     I created a directory for a file that had just the first 100 lines
>     of Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt
>     (/home/SGC/smr/data/intermine/simrmine/flyexpression/flyexpression.txt)
>
>     *I added this to my project.xml:*
>          <source name="flybase-expression" type="flybase-expression">
>            <property name="src.data.dir"
>     location="/home/SGC/smr/data/intermine/simrmine/flyexpression/"/>
>          </source>
>
>     *In my integrate directory I ran*
>     ant -Dsource=flybase-expression -v from
>
>
>     *and i get this error:*
>
>     -retrieve-tgt-from-custom-file:
>     [integrate] Processing file:
>     /home/SGC/smr/data/intermine/simrmine/flyexpression/flyexpression.txt
>     [integrate] Creating id resolver from database and caching id
>     resolver to file: build/idresolver.cache
>           [null] Exiting
>     /var/lib/pgsql/other_data/intermine/bio/sources/fly/flybase-expression/build.xml.
>
>     BUILD FAILED
>     /var/lib/pgsql/other_data/intermine/imbuild/integrate.xml:54: The
>     following error occurred while executing this line:
>     /var/lib/pgsql/other_data/intermine/imbuild/source.xml:145:
>     java.lang.NullPointerException
>     at java.io.FileInputStream.<init>(FileInputStream.java:138)
>     at java.io.FileReader.<init>(FileReader.java:72)
>     at
>     org.intermine.bio.dataconversion.FlybaseExpressionConverter.process(FlybaseExpressionConverter.java:101)
>     at
>     org.intermine.task.FileConverterTask.execute(FileConverterTask.java:116)
>     at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
>     at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>     at
>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at
>     org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>     at org.apache.tools.ant.Task.perform(Task.java:348)
>     at org.apache.tools.ant.Target.execute(Target.java:435)
>     at org.apache.tools.ant.Target.performTasks(Target.java:456)
>     at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
>     at
>     org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
>     at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
>     at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:440)
>     at org.intermine.task.Integrate.performAction(Integrate.java:223)
>     at org.intermine.task.Integrate.performAction(Integrate.java:135)
>     at org.intermine.task.Integrate.execute(Integrate.java:127)
>     at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
>     at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>     at
>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at
>     org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>     at org.apache.tools.ant.Task.perform(Task.java:348)
>     at org.apache.tools.ant.Target.execute(Target.java:435)
>     at org.apache.tools.ant.Target.performTasks(Target.java:456)
>     at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
>     at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
>     at
>     org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
>     at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
>     at org.apache.tools.ant.Main.runBuild(Main.java:851)
>     at org.apache.tools.ant.Main.startAnt(Main.java:235)
>     at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
>     at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
>
>
>     On Fri, Nov 6, 2015 at 2:49 AM, sergio contrino
>     <[hidden email] <mailto:[hidden email]>> wrote:
>
>         dear sofia,
>         you can see the code related to the expression data in this
>         repository
>         https://github.com/modENCODE-DCC/modmine
>         more specifically
>
>         https://github.com/modENCODE-DCC/modmine/tree/dev/bio/sources/modmine/fly-expression-score
>
>         an example of the data used is below, and you can wget the full
>         file here
>
>         http://www.flymine.org/download/modmine/misc/Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt
>
>         the display is done with an old canvasXpress, you could probably
>         use the updated version or switch to something else.
>         i hope this helps
>         thank you
>         sergio
>
>
>
>         Gene_FBgn       ANNOTATION_SYMBOL       NAME    SYMBOL
>         Chromosome CME_L1  Sg4     ML-DmD11        ML-DmD20-c2
>           ML-DmD20-c5     Kc167 GM2     S2-DRSC S2R+    S1      1182-4H
>         ML-DmD16-c3     ML-DmD32 ML-DmD17-c3     ML-DmD8 CME W1 cl.8+
>         ML-DmD9 ML-DmBG1-c1     ML-DmD21        ML-DmD4-c1
>         ML-DmBG3-c2     S3      CME W2  mbn2 ML-DmBG2-c2     emb_0-2h
>              emb_2-4h        emb_4-6h        emb_6-8h        emb_8-10h
>               emb_10-12h      emb_12-14h      emb_14-16h emb_16-18h
>         emb_18-20h      emb_20-22h      emb_22-24h L1_larvae
>           L2_larvae       L3_larvae_12hr  L3_larvae_dark_blue
>         L3_larvae_light_blue    L3_larvae_clear WPP     WPP_12hr
>         WPP_24hr        WPP_2days       WPP_3days       WPP_4days
>           Adult_F_Ecl_1day        Adult_M_Ecl_1day
>         Adult_F_Ecl_5day        Adult_M_Ecl_5day
>         Adult_F_Ecl_30day       Adult_M_Ecl_30day
>         FBgn0000003     CR32864 RNA 7SL 7SLRNA  chr3R   2865    766
>           1081 2554    6007    28853   15965   19318   40438   12516
>           11627   6284 21483   16001   86      28398   9037    19780
>           18895   6302    0 11805   11449   8065    7327    3169
>         19549   18903   0       742 2041    2104    10655   2660
>         11762   16171   9668    12804   13167 10390   6637    4434
>         3346    0       4852    0       2184    7606 8059    8887
>         6918    2544    5340    5663    1474
>         FBgn0000008     CG6741  arc     a       chr2R   207     131
>           349 246     370     292     103     179     128     200
>           343     239 211     164     519     336     110     397
>           303     210     180 57      140     149     204     1069
>         461     284     280     384 586     689     591     262     348
>             286     310     201     286 247     181     227     415
>           556     487     757     1027    630 366     320     269
>           430     219     359     273
>         FBgn0000014     CG10325 abdominal A     abd-A   chr3R   176
>           91 90      143     114     89      43      87      144     92
>              37 54      106     115     32      55      76      271
>           101     112 64      223     57      26      88      55
>         183     647     984 1018    779     636     627     435     322
>             243     331     290 311     334     536     625     692
>           888     460     503     346 212     156     169     148
>           58      81      120     86
>         FBgn0000015     CG11648 Abdominal B     Abd-B   chr3R   645
>           1172 101     95      86      99      62      67      111
>           45      117 76      57      89      117     120     69
>         104     79      28 60      1709    41      48      54      101
>             136     574     474 566     281     348     239     189
>           154     152     144     157 81      152     89      106
>           143     96      63      104     97 86      59      75      74
>              39      73      48      78
>         FBgn0000017     CG4032  Abl tyrosine kinase     Abl     chr3L
>           2076 1216    1315    1402    2413    1184    2013    1583
>         2253    1758 1802    1252    3079    1169    2281    1487
>         3399    1036    4081 1864    2052    1026    1791    1308
>         899     2711    2307    1093 958     1010    1564    1334
>         948     687     706     862     614 591     483     474     740
>             518     830     854     699     662 990     431     407
>           618     336     768     513     637     511
>         FBgn0000018     CG6093  abnormal oocyte abo     chr2L   166
>           264 244     313     266     213     208     305     364
>           269     233 163     175     158     184     191     230
>           330     235     267 178     172     236     490     198
>           627     11      88      68 29      30      33      15      27
>              33      47      14      15 27      37      56      88
>         51      58      29      37      47 15      22      87      60
>            235     65      132     40
>         FBgn0000022     CG3796  achaete ac      chrX    1       0
>           6 0       8       27      5       27      31      0       352
>             49 25      0       20      0       0       59      30
>         3       40 4       0       7       0       69      465     1018
>            482     302 40      0       10      0       14      45
>         19      13      1 28      47      52      46      61      85
>            69      6       0 3       18      9       31      20      18
>              10
>         FBgn0000024     CG17907 Acetylcholine esterase  Ace     chr3R
>           2471 714     2119    2049    753     106     547     1404
>         84      96 223     394     1984    196     836     321     149
>             32      431 1530    801     2478    57      56      56
>         287     51      63 67      34      103     147     408     365
>             436     561     550 551     528     603     156     221
>           265     349     456     316 500     670     530     401
>           416     136     377     123     435
>         FBgn0000028     CG9151  abnormal chemosensory jump 6    acj6
>         chrX 84      111     171     146     146     187     83
>         122     253 96      92      115     132     89      109     332
>             196     144 131     76      169     98      184     112
>           204     106     135 302     139     63      54      132
>           164     150     96      140 76      78      77      144
>           197     65      55      86      60 169     110     126
>           130     89      91      92      81      135     155
>
>
>
>         On 05/11/15 20:00, Sofia Robb wrote:
>
>             Can some one send me a sample of how you load differential
>             expression
>             data from different experiments into intermine? A small
>             sample of your
>             intermine xml or what ever file type you use and your datasource
>             information would be very very helpful. modencode has some
>             very nice
>             pages with differential expression data that I would love to
>             replicate.
>             http://intermine.modencode.org/release-33/bagDetails.do?scope=global&bagName=example
>
>             Thank you very much,
>             Sofia
>
>
>             _______________________________________________
>             dev mailing list
>             [hidden email] <mailto:[hidden email]>
>             http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>
>
>         --
>         sergio contrino                  InterMine, University of Cambridge
>         https://sergiocontrino.github.io http://www.intermine.org
>
>
>

--
sergio contrino                  InterMine, University of Cambridge
https://sergiocontrino.github.io           http://www.intermine.org

_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: differential expression

Sofia Robb-4
I will try this. No need to follow up with my first message about errors.

Thanks!!!

Sent from my iPhone

> On Nov 10, 2015, at 3:08 AM, sergio contrino <[hidden email]> wrote:
>
> dear sofia,
> apologies for the slow response. most of the model modification for modencode are in the file ~(in the usual repository)
>
> https://github.com/modENCODE-DCC/modmine/blob/dev/modmine/dbmodel/resources/modencode-metadata_additions.xml
>
> so yes, you need to adapt the loader to your model. for example you probably don't need to deal with submissions.
>
> please let me know if this yours does not supersede your previous mail and i'll have a look at it today.
>
> thank you!
> sergio
>
>
>
>> On 09/11/15 18:48, Sofia Robb wrote:
>> I switched my data source to the modmine fly-expression-score. I think
>> this is the one I should use. And I get much more informative errors.
>> From these errors, I think I need the
>> modmine/fly-expression-score_additions.xml file. The one I have is has
>> no classes defined.  I think it would be good to get the fly data to
>> load, then I can modify my data and FlyExpressionScoreConverter.java to
>> fit my data.
>>
>> Caused by: java.lang.IllegalArgumentException: class
>> "org.intermine.model.bio.Submission" is not in the Model
>>
>>
>> BUILD FAILED
>> /var/lib/pgsql/other_data/intermine/imbuild/integrate.xml:54: The
>> following error occurred while executing this line:
>> /var/lib/pgsql/other_data/intermine/imbuild/source.xml:145:
>> java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> at
>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>> at org.intermine.task.FileConverterTask.execute(FileConverterTask.java:97)
>> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>> at org.apache.tools.ant.Task.perform(Task.java:348)
>> at org.apache.tools.ant.Target.execute(Target.java:435)
>> at org.apache.tools.ant.Target.performTasks(Target.java:456)
>> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
>> at
>> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
>> at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
>> at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:440)
>> at org.intermine.task.Integrate.performAction(Integrate.java:223)
>> at org.intermine.task.Integrate.performAction(Integrate.java:135)
>> at org.intermine.task.Integrate.execute(Integrate.java:127)
>> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>> at org.apache.tools.ant.Task.perform(Task.java:348)
>> at org.apache.tools.ant.Target.execute(Target.java:435)
>> at org.apache.tools.ant.Target.performTasks(Target.java:456)
>> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
>> at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
>> at
>> org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
>> at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
>> at org.apache.tools.ant.Main.runBuild(Main.java:851)
>> at org.apache.tools.ant.Main.startAnt(Main.java:235)
>> at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
>> at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
>> Caused by: java.lang.IllegalArgumentException: class
>> "org.intermine.model.bio.Submission" is not in the Model
>> at org.intermine.xml.full.Item.getClassDescriptorByName(Item.java:546)
>> at org.intermine.xml.full.Item.setClassName(Item.java:128)
>> at org.intermine.xml.full.ItemFactory.makeItem(ItemFactory.java:114)
>> at
>> org.intermine.dataconversion.DataConverter.createItem(DataConverter.java:104)
>> at
>> org.intermine.bio.dataconversion.FlyExpressionScoreConverter.createSubmissionItem(FlyExpressionScoreConverter.java:414)
>> at
>> org.intermine.bio.dataconversion.FlyExpressionScoreConverter.<init>(FlyExpressionScoreConverter.java:65)
>> ... 36 more
>>
>> On Fri, Nov 6, 2015 at 12:55 PM, Sofia Robb <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>    Ok, I tried to load the above file but failed.
>>
>>    I created a directory for a file that had just the first 100 lines
>>    of Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt
>>    (/home/SGC/smr/data/intermine/simrmine/flyexpression/flyexpression.txt)
>>
>>    *I added this to my project.xml:*
>>         <source name="flybase-expression" type="flybase-expression">
>>           <property name="src.data.dir"
>>    location="/home/SGC/smr/data/intermine/simrmine/flyexpression/"/>
>>         </source>
>>
>>    *In my integrate directory I ran*
>>    ant -Dsource=flybase-expression -v from
>>
>>
>>    *and i get this error:*
>>
>>    -retrieve-tgt-from-custom-file:
>>    [integrate] Processing file:
>>    /home/SGC/smr/data/intermine/simrmine/flyexpression/flyexpression.txt
>>    [integrate] Creating id resolver from database and caching id
>>    resolver to file: build/idresolver.cache
>>          [null] Exiting
>>    /var/lib/pgsql/other_data/intermine/bio/sources/fly/flybase-expression/build.xml.
>>
>>    BUILD FAILED
>>    /var/lib/pgsql/other_data/intermine/imbuild/integrate.xml:54: The
>>    following error occurred while executing this line:
>>    /var/lib/pgsql/other_data/intermine/imbuild/source.xml:145:
>>    java.lang.NullPointerException
>>    at java.io.FileInputStream.<init>(FileInputStream.java:138)
>>    at java.io.FileReader.<init>(FileReader.java:72)
>>    at
>>    org.intermine.bio.dataconversion.FlybaseExpressionConverter.process(FlybaseExpressionConverter.java:101)
>>    at
>>    org.intermine.task.FileConverterTask.execute(FileConverterTask.java:116)
>>    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
>>    at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>    at
>>    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    at java.lang.reflect.Method.invoke(Method.java:606)
>>    at
>>    org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>>    at org.apache.tools.ant.Task.perform(Task.java:348)
>>    at org.apache.tools.ant.Target.execute(Target.java:435)
>>    at org.apache.tools.ant.Target.performTasks(Target.java:456)
>>    at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
>>    at
>>    org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
>>    at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
>>    at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:440)
>>    at org.intermine.task.Integrate.performAction(Integrate.java:223)
>>    at org.intermine.task.Integrate.performAction(Integrate.java:135)
>>    at org.intermine.task.Integrate.execute(Integrate.java:127)
>>    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
>>    at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>    at
>>    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>    at java.lang.reflect.Method.invoke(Method.java:606)
>>    at
>>    org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>>    at org.apache.tools.ant.Task.perform(Task.java:348)
>>    at org.apache.tools.ant.Target.execute(Target.java:435)
>>    at org.apache.tools.ant.Target.performTasks(Target.java:456)
>>    at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
>>    at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
>>    at
>>    org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
>>    at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
>>    at org.apache.tools.ant.Main.runBuild(Main.java:851)
>>    at org.apache.tools.ant.Main.startAnt(Main.java:235)
>>    at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
>>    at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
>>
>>
>>    On Fri, Nov 6, 2015 at 2:49 AM, sergio contrino
>>    <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>        dear sofia,
>>        you can see the code related to the expression data in this
>>        repository
>>        https://github.com/modENCODE-DCC/modmine
>>        more specifically
>>
>>        https://github.com/modENCODE-DCC/modmine/tree/dev/bio/sources/modmine/fly-expression-score
>>
>>        an example of the data used is below, and you can wget the full
>>        file here
>>
>>        http://www.flymine.org/download/modmine/misc/Drosophila_Cell_Lines_and_Developmental_Stages_Gene_Scores.txt
>>
>>        the display is done with an old canvasXpress, you could probably
>>        use the updated version or switch to something else.
>>        i hope this helps
>>        thank you
>>        sergio
>>
>>
>>
>>        Gene_FBgn       ANNOTATION_SYMBOL       NAME    SYMBOL
>>        Chromosome CME_L1  Sg4     ML-DmD11        ML-DmD20-c2
>>          ML-DmD20-c5     Kc167 GM2     S2-DRSC S2R+    S1      1182-4H
>>        ML-DmD16-c3     ML-DmD32 ML-DmD17-c3     ML-DmD8 CME W1 cl.8+
>>        ML-DmD9 ML-DmBG1-c1     ML-DmD21        ML-DmD4-c1
>>        ML-DmBG3-c2     S3      CME W2  mbn2 ML-DmBG2-c2     emb_0-2h
>>             emb_2-4h        emb_4-6h        emb_6-8h        emb_8-10h
>>              emb_10-12h      emb_12-14h      emb_14-16h emb_16-18h
>>        emb_18-20h      emb_20-22h      emb_22-24h L1_larvae
>>          L2_larvae       L3_larvae_12hr  L3_larvae_dark_blue
>>        L3_larvae_light_blue    L3_larvae_clear WPP     WPP_12hr
>>        WPP_24hr        WPP_2days       WPP_3days       WPP_4days
>>          Adult_F_Ecl_1day        Adult_M_Ecl_1day
>>        Adult_F_Ecl_5day        Adult_M_Ecl_5day
>>        Adult_F_Ecl_30day       Adult_M_Ecl_30day
>>        FBgn0000003     CR32864 RNA 7SL 7SLRNA  chr3R   2865    766
>>          1081 2554    6007    28853   15965   19318   40438   12516
>>          11627   6284 21483   16001   86      28398   9037    19780
>>          18895   6302    0 11805   11449   8065    7327    3169
>>        19549   18903   0       742 2041    2104    10655   2660
>>        11762   16171   9668    12804   13167 10390   6637    4434
>>        3346    0       4852    0       2184    7606 8059    8887
>>        6918    2544    5340    5663    1474
>>        FBgn0000008     CG6741  arc     a       chr2R   207     131
>>          349 246     370     292     103     179     128     200
>>          343     239 211     164     519     336     110     397
>>          303     210     180 57      140     149     204     1069
>>        461     284     280     384 586     689     591     262     348
>>            286     310     201     286 247     181     227     415
>>          556     487     757     1027    630 366     320     269
>>          430     219     359     273
>>        FBgn0000014     CG10325 abdominal A     abd-A   chr3R   176
>>          91 90      143     114     89      43      87      144     92
>>             37 54      106     115     32      55      76      271
>>          101     112 64      223     57      26      88      55
>>        183     647     984 1018    779     636     627     435     322
>>            243     331     290 311     334     536     625     692
>>          888     460     503     346 212     156     169     148
>>          58      81      120     86
>>        FBgn0000015     CG11648 Abdominal B     Abd-B   chr3R   645
>>          1172 101     95      86      99      62      67      111
>>          45      117 76      57      89      117     120     69
>>        104     79      28 60      1709    41      48      54      101
>>            136     574     474 566     281     348     239     189
>>          154     152     144     157 81      152     89      106
>>          143     96      63      104     97 86      59      75      74
>>             39      73      48      78
>>        FBgn0000017     CG4032  Abl tyrosine kinase     Abl     chr3L
>>          2076 1216    1315    1402    2413    1184    2013    1583
>>        2253    1758 1802    1252    3079    1169    2281    1487
>>        3399    1036    4081 1864    2052    1026    1791    1308
>>        899     2711    2307    1093 958     1010    1564    1334
>>        948     687     706     862     614 591     483     474     740
>>            518     830     854     699     662 990     431     407
>>          618     336     768     513     637     511
>>        FBgn0000018     CG6093  abnormal oocyte abo     chr2L   166
>>          264 244     313     266     213     208     305     364
>>          269     233 163     175     158     184     191     230
>>          330     235     267 178     172     236     490     198
>>          627     11      88      68 29      30      33      15      27
>>             33      47      14      15 27      37      56      88
>>        51      58      29      37      47 15      22      87      60
>>           235     65      132     40
>>        FBgn0000022     CG3796  achaete ac      chrX    1       0
>>          6 0       8       27      5       27      31      0       352
>>            49 25      0       20      0       0       59      30
>>        3       40 4       0       7       0       69      465     1018
>>           482     302 40      0       10      0       14      45
>>        19      13      1 28      47      52      46      61      85
>>           69      6       0 3       18      9       31      20      18
>>             10
>>        FBgn0000024     CG17907 Acetylcholine esterase  Ace     chr3R
>>          2471 714     2119    2049    753     106     547     1404
>>        84      96 223     394     1984    196     836     321     149
>>            32      431 1530    801     2478    57      56      56
>>        287     51      63 67      34      103     147     408     365
>>            436     561     550 551     528     603     156     221
>>          265     349     456     316 500     670     530     401
>>          416     136     377     123     435
>>        FBgn0000028     CG9151  abnormal chemosensory jump 6    acj6
>>        chrX 84      111     171     146     146     187     83
>>        122     253 96      92      115     132     89      109     332
>>            196     144 131     76      169     98      184     112
>>          204     106     135 302     139     63      54      132
>>          164     150     96      140 76      78      77      144
>>          197     65      55      86      60 169     110     126
>>          130     89      91      92      81      135     155
>>
>>
>>
>>        On 05/11/15 20:00, Sofia Robb wrote:
>>
>>            Can some one send me a sample of how you load differential
>>            expression
>>            data from different experiments into intermine? A small
>>            sample of your
>>            intermine xml or what ever file type you use and your datasource
>>            information would be very very helpful. modencode has some
>>            very nice
>>            pages with differential expression data that I would love to
>>            replicate.
>>            http://intermine.modencode.org/release-33/bagDetails.do?scope=global&bagName=example
>>
>>            Thank you very much,
>>            Sofia
>>
>>
>>            _______________________________________________
>>            dev mailing list
>>            [hidden email] <mailto:[hidden email]>
>>            http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>>
>>
>>        --
>>        sergio contrino                  InterMine, University of Cambridge
>>        https://sergiocontrino.github.io http://www.intermine.org
>
> --
> sergio contrino                  InterMine, University of Cambridge
> https://sergiocontrino.github.io           http://www.intermine.org

_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev