update-publications job failing

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

update-publications job failing

De Pons, Jeffrey

Hello,

 

I am updating our mine to version 1.7.1 and am having trouble getting the update-publications job to complete.  It appears some bad characters are being retrieved via eutils and then loaded into Postgres.   Has anybody else run into this issue?

 

Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0xed 0xa0 0xb5

 

Here is the full stack trace.

 

starting command: ant -v -Dsource=update-publications

 

BUILD FAILED

/data/intermine/git_ratmine/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:

/data/intermine/git_ratmine/intermine/imbuild/source.xml:219: org.intermine.objectstore.ObjectStoreException: Error committing transaction

        at org.intermine.objectstore.intermine.ObjectStoreWriterInterMineImpl.commitTransactionWithConnection(ObjectStoreWriterInterMineImpl.java:1572)

        at org.intermine.objectstore.intermine.ObjectStoreWriterInterMineImpl.commitTransaction(ObjectStoreWriterInterMineImpl.java:1547)

        at org.intermine.dataconversion.ObjectStoreItemWriter.close(ObjectStoreItemWriter.java:130)

        at org.intermine.task.FullXmlConverterTask.execute(FullXmlConverterTask.java:140)

        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)

        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)

        at org.apache.tools.ant.Task.perform(Task.java:348)

        at org.apache.tools.ant.Target.execute(Target.java:435)

        at org.apache.tools.ant.Target.performTasks(Target.java:456)

        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405)

        at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)

        at org.apache.tools.ant.Project.executeTargets(Project.java:1260)

        at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:441)

        at org.intermine.task.Integrate.performAction(Integrate.java:219)

        at org.intermine.task.Integrate.performAction(Integrate.java:131)

        at org.intermine.task.Integrate.execute(Integrate.java:123)

        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)

        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)

        at org.apache.tools.ant.Task.perform(Task.java:348)

        at org.apache.tools.ant.Target.execute(Target.java:435)

        at org.apache.tools.ant.Target.performTasks(Target.java:456)

        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405)

        at org.apache.tools.ant.Project.executeTarget(Project.java:1376)

        at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)

        at org.apache.tools.ant.Project.executeTargets(Project.java:1260)

        at org.apache.tools.ant.Main.runBuild(Main.java:853)

        at org.apache.tools.ant.Main.startAnt(Main.java:235)

        at org.apache.tools.ant.launch.Launcher.run(Launcher.java:285)

        at org.apache.tools.ant.launch.Launcher.main(Launcher.java:112)

Caused by: java.sql.SQLException: Error writing to database, running statement COPY Item (implementations, className, identifier, id) FROM STDIN BINARY, data size = 401709

        at org.intermine.sql.writebatch.FlushJobPostgresCopyImpl.flush(FlushJobPostgresCopyImpl.java:56)

        at org.intermine.sql.writebatch.Batch$BatchFlusher.run(Batch.java:461)

        at java.lang.Thread.run(Thread.java:745)

Caused by: org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block

        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)

        at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1049)

        at org.postgresql.core.v3.QueryExecutorImpl.startCopy(QueryExecutorImpl.java:766)

        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:48)

        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:209)

        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:192)

        at org.intermine.sql.writebatch.FlushJobPostgresCopyImpl.flush(FlushJobPostgresCopyImpl.java:51)

        ... 2 more

Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0xed 0xa0 0xb5

  Where: COPY attribute, line 51855, column intermine_value

        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)

        at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1049)

        at org.postgresql.core.v3.QueryExecutorImpl.endCopy(QueryExecutorImpl.java:892)

        at org.postgresql.core.v3.CopyInImpl.endCopy(CopyInImpl.java:40)

        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:216)

        ... 4 more

 

Total time: 32 minutes 55 seconds

Mon Apr 10 13:42:50 CDT 2017

 

finished

 

 

ERROR: 256

failed with exit code 0: ant -v -Dsource=update-publications


_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: update-publications job failing

Sam Hokin-3
Hi, Jeff! I think most of us are probably using SQL_ASCII encoding in our databases, in which case I don't think there are any
checks on encoding and you wouldn't throw that PSQLException.

IM is supposed to support UTF-8 encoding (I think) but they recommend SQL_ASCII for performance.

http://intermine.readthedocs.io/en/latest/system-requirements/software/postgres/postgres/

On 04/10/2017 01:03 PM, De Pons, Jeffrey wrote:
> Hello,
>
> I am updating our mine to version 1.7.1 and am having trouble getting the update-publications job to complete.  It appears some bad
> characters are being retrieved via eutils and then loaded into Postgres.   Has anybody else run into this issue?
>
> *Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0xed 0xa0 0xb5*

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Loading...