Anyone seen this invalid byte sequence error message?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Anyone seen this invalid byte sequence error message?

joe carlson
Hi folks,

I’ve been seeing this error message sporadically with my latest build:

BUILD FAILED
/global/u1/j/jcarlson/src/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
/global/u1/j/jcarlson/src/intermine/imbuild/source.xml:146: org.intermine.objectstore.ObjectStoreException: Error committing transaction
        at org.intermine.objectstore.intermine.ObjectStoreWriterInterMineImpl.commitTransactionWithConnection(ObjectStoreWriterInterMineImpl.java:1572)
        at org.intermine.objectstore.intermine.ObjectStoreWriterInterMineImpl.commitTransaction(ObjectStoreWriterInterMineImpl.java:1547)              
        at org.intermine.dataconversion.ObjectStoreItemWriter.close(ObjectStoreItemWriter.java:130)                                                    


<snip>

Caused by: java.sql.SQLException: Error writing to database, running statement COPY Item (implementations, className, identifier, id) FROM STDIN BINARY, data size = 736983
        at org.intermine.sql.writebatch.FlushJobPostgresCopyImpl.flush(FlushJobPostgresCopyImpl.java:58)
        at org.intermine.sql.writebatch.Batch$BatchFlusher.run(Batch.java:461)
        at java.lang.Thread.run(Thread.java:744)
Caused by: org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)
        at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1049)
        at org.postgresql.core.v3.QueryExecutorImpl.startCopy(QueryExecutorImpl.java:766)
        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:48)
        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:209)
        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:192)
        at org.intermine.sql.writebatch.FlushJobPostgresCopyImpl.flush(FlushJobPostgresCopyImpl.java:53)
        ... 2 more
Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Where: COPY attribute, line 7919, column intermine_value
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)
        at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1049)
        at org.postgresql.core.v3.QueryExecutorImpl.writeToCopy(QueryExecutorImpl.java:926)
        at org.postgresql.core.v3.CopyInImpl.writeToCopy(CopyInImpl.java:32)
        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:213)

It seems that there are invalid byte sequence when writing to the ‘items’ database. I’m seeing this with my most recent build for a few different data loaders; the interpro loader being one of them. There are no invalid characters in the input files that I can see.

I’ve recently done a merge with upstream for this build, but otherwise have not touched the java code. Is it a problem with the postgres jdbc? My attempts at debugging have not shown anything. Has anyone experienced this before?

Thanks,

Joe

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Anyone seen this invalid byte sequence error message?

joe carlson
Yeah. nevermind.

I had some ISO-8859 characters in this file. I thought I had processed it before; maybe I did by stripping those characters out seems to help things.

Joe

On Mar 24, 2018, at 4:01 PM, Joe Carlson <[hidden email]> wrote:

Hi folks,

I’ve been seeing this error message sporadically with my latest build:

BUILD FAILED
/global/u1/j/jcarlson/src/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
/global/u1/j/jcarlson/src/intermine/imbuild/source.xml:146: org.intermine.objectstore.ObjectStoreException: Error committing transaction
        at org.intermine.objectstore.intermine.ObjectStoreWriterInterMineImpl.commitTransactionWithConnection(ObjectStoreWriterInterMineImpl.java:1572)
        at org.intermine.objectstore.intermine.ObjectStoreWriterInterMineImpl.commitTransaction(ObjectStoreWriterInterMineImpl.java:1547)              
        at org.intermine.dataconversion.ObjectStoreItemWriter.close(ObjectStoreItemWriter.java:130)                                                    


<snip>

Caused by: java.sql.SQLException: Error writing to database, running statement COPY Item (implementations, className, identifier, id) FROM STDIN BINARY, data size = 736983
        at org.intermine.sql.writebatch.FlushJobPostgresCopyImpl.flush(FlushJobPostgresCopyImpl.java:58)
        at org.intermine.sql.writebatch.Batch$BatchFlusher.run(Batch.java:461)
        at java.lang.Thread.run(Thread.java:744)
Caused by: org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)
        at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1049)
        at org.postgresql.core.v3.QueryExecutorImpl.startCopy(QueryExecutorImpl.java:766)
        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:48)
        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:209)
        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:192)
        at org.intermine.sql.writebatch.FlushJobPostgresCopyImpl.flush(FlushJobPostgresCopyImpl.java:53)
        ... 2 more
Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Where: COPY attribute, line 7919, column intermine_value
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)
        at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1049)
        at org.postgresql.core.v3.QueryExecutorImpl.writeToCopy(QueryExecutorImpl.java:926)
        at org.postgresql.core.v3.CopyInImpl.writeToCopy(CopyInImpl.java:32)
        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:213)

It seems that there are invalid byte sequence when writing to the ‘items’ database. I’m seeing this with my most recent build for a few different data loaders; the interpro loader being one of them. There are no invalid characters in the input files that I can see.

I’ve recently done a merge with upstream for this build, but otherwise have not touched the java code. Is it a problem with the postgres jdbc? My attempts at debugging have not shown anything. Has anyone experienced this before?

Thanks,

Joe


_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev