 |
 |
 |
 |
Post questions or issues to the Ebean google group - https://groups.google.com/forum/#!forum/ebean
Maven central - all related projects
- ebean-agent
- ebean-maven-pluginI'm using Play 2.2.1, which includes ebeanorm-3.2.2 which I tested against and found (what I believe) this bug.
I am getting PersistenceExceptions thrown when creating a large set (hundreds) of similar queries in a row. It seems to depend on the data I'm passing in, which unfortunately, I cannot disclose. The queries look very similar, but have varying numbers of parameters in each section. I traced this through the ebean source code (which I repackaged and replaced the jar in my Play repo), and it lead me to believe that the hashes were colliding. I printed the computed hashes of each query, and they did indeed match. It seems that because they hash the same, the same QueryPlan object it being used for both, even though they have differing numbers of bound parameters.
Both below errors hash to the value: 1336939366
Before I knew all of this, I attempted to explain what I was seeing over on stackoverflow if you'd like to take a look at that as well. http://stackoverflow.com/questions/20501591/ebean-dynamic-query-prepared-statements-mismatched-parameter-count-error
Here are the two stack traces:
play.api.Application$$anon$1: Execution exception[[PersistenceException: Query threw SQLException:No value specified for parameter 19
Bind values:[SENT,1290,8988,13032,13052,1290,96,99,100,101,102,103,104,105,106,107,108,222]
Query was:
select distinct t0.contact_id c0, t0.contact_uuid c1, t0.bounce c2 from contact t0 join email_record u1 on u1.contact_id = t0.contact_id join contact_tag u2z_ on u2z_.contact_id = t0.contact_id join tag u2 on u2.tag_id = u2z_.tag_id where u1.status = ? and t0.unit_id = ? and u2.tag_id in (?,?) and t0.unit_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? ) and t0.campaign_id in (?,?,?,?,?,?,?,?,?,?,?,?)
]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10.jar:2.2.1]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
at scala.Option.map(Option.scala:145) [scala-library.jar:na]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2.applyOrElse(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
Caused by: javax.persistence.PersistenceException: Query threw SQLException:No value specified for parameter 19
Bind values:[SENT,1290,8988,13032,13052,1290,96,99,100,101,102,103,104,105,106,107,108,222]
Query was:
select distinct t0.contact_id c0, t0.contact_uuid c1, t0.bounce c2 from contact t0 join email_record u1 on u1.contact_id = t0.contact_id join contact_tag u2z_ on u2z_.contact_id = t0.contact_id join tag u2 on u2.tag_id = u2z_.tag_id where u1.status = ? and t0.unit_id = ? and u2.tag_id in (?,?) and t0.unit_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? ) and t0.campaign_id in (?,?,?,?,?,?,?,?,?,?,?,?)
at com.avaje.ebeaninternal.server.query.CQuery.createPersistenceException(CQuery.java:798) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.query.CQuery.createPersistenceException(CQuery.java:775) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.query.CQueryEngine.findMany(CQueryEngine.java:205) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.query.DefaultOrmQueryEngine.findMany(DefaultOrmQueryEngine.java:81) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.core.OrmQueryRequest.findList(OrmQueryRequest.java:265) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.core.DefaultServer.findList(DefaultServer.java:1469) ~[avaje-ebeanorm.jar:na]
Caused by: java.sql.SQLException: No value specified for parameter 19
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.PreparedStatement.checkAllParametersSet(PreparedStatement.java:2612) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.PreparedStatement.fillSendPacket(PreparedStatement.java:2587) ~[mysql-connector-java-5.1.27.jar:na]
And the second:
play.api.Application$$anon$1: Execution exception[[PersistenceException: Error with property[19] dt[4]data[1464][java.lang.Integer]]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10.jar:2.2.1]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
at scala.Option.map(Option.scala:145) [scala-library.jar:na]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2.applyOrElse(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
Caused by: javax.persistence.PersistenceException: Error with property[19] dt[4]data[1464][java.lang.Integer]
at com.avaje.ebeaninternal.server.persist.Binder.bindSimpleData(Binder.java:327) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.persist.Binder.bindObject(Binder.java:216) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.persist.Binder.bindObject(Binder.java:173) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.query.CQueryPredicates.bind(CQueryPredicates.java:152) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.query.CQuery.prepareBindExecuteQuery(CQuery.java:382) ~[avaje-ebeanorm.jar:na]
at com.avaje.ebeaninternal.server.query.CQueryEngine.findMany(CQueryEngine.java:165) ~[avaje-ebeanorm.jar:na]
Caused by: java.sql.SQLException: Parameter index out of range (19 > number of parameters, which is 18).
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.PreparedStatement.checkBounds(PreparedStatement.java:3814) ~[mysql-connector-java-5.1.27.jar:na]
at com.mysql.jdbc.PreparedStatement.setInternal(PreparedStatement.java:3796) ~[mysql-connector-java-5.1.27.jar:na]
This is affecting our production environment so we need a fix ASAP, so I'm going to attempt to fix this in the ebean source, and if I can, I'll try to submit a patch to this ticket. However, I am very unfamiliar with this code, so if someone has the time I'd really appreciate the help.
I can provide more data on request. Additionally, I submitted this bug on what I assume to be the old bug system before seeing the dates on the issues there, http://www.avaje.org/bug.html.
class Model {
String a;
@Transient
List<ModelB> list;
}
Model model = findById(1);
model.list = find(ModelB.class).select("oneField").findList();
//path -> (a,list(oneField))
jsonContext.toJson(o, generator, path)
//-> {"a":"a",list:[{"oneField":1,"twoField":null}]} should not show `twoField`
Ebean version: 7.3.1 Ebean datasource version: 1.1.1
Reproduce steps:
javax.persistence.PersistenceException: java.sql.SQLException: Interrupted getting connection from pool java.lang.InterruptedException
at com.avaje.ebeaninternal.server.core.DefaultContainer.checkDataSource(DefaultContainer.java:361)
at com.avaje.ebeaninternal.server.core.DefaultContainer.createServer(DefaultContainer.java:102)
at com.avaje.ebeaninternal.server.core.DefaultContainer.createServer(DefaultContainer.java:41)
at com.avaje.ebean.EbeanServerFactory.createInternal(EbeanServerFactory.java:108)
at com.avaje.ebean.EbeanServerFactory.create(EbeanServerFactory.java:67)
at act.db.ebean.EbeanService$2.on(EbeanService.java:61)
at act.event.EventBus.callOn(EventBus.java:134)
at act.event.EventBus.callOn(EventBus.java:155)
at act.event.EventBus.callOn(EventBus.java:171)
at act.event.EventBus.emit(EventBus.java:189)
at act.event.EventBus.emit(EventBus.java:180)
at act.app.App.emit(App.java:504)
at act.app.App.refresh(App.java:273)
at act.app.CliSession.run(CliSession.java:90)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: Interrupted getting connection from pool java.lang.InterruptedException
at org.avaje.datasource.pool.PooledConnectionQueue.getPooledConnection(PooledConnectionQueue.java:277)
at org.avaje.datasource.pool.ConnectionPool.getPooledConnection(ConnectionPool.java:712)
at org.avaje.datasource.pool.ConnectionPool.getConnection(ConnectionPool.java:700)
at com.avaje.ebeaninternal.server.core.DefaultContainer.checkDataSource(DefaultContainer.java:351)
... 20 more
Set the index of the first row of the results to return.
SqlQuery.setFirstRow(10) findList return first row index 0
如果有一张订单表, 需要将状态由已创建变更为已支付, 对应的 sql 语句是:
update t_order set status = 'Pay' where id = xxx and status = 'Create'
然后根据返回的结果是否为 1 来判断更新有没有成功, 从而决定是否将事务进行回滚.
不想使用 version 的原因在于: 如果有其他的地方在做跟 status 无关的更新操作, 这里的更新将会失败, 还是想基于这一个属性做这种颗粒度的更新, 而且从性能的角度来说, 也多了一次 load version 的查询
v = select version from t_order where id = xxx
update t_order set status = 'Pay', version = v + 1
where id = xxx and version = v
虽然也能用 createQuery 的方式完成这个实现
public void pay(Long id) {
String delete = "update t_order set status = :new and id = :id and status = :old";
SqlUpdate update = ebeanServer.createSqlUpdate(delete);
update.setParameter("id", id);
update.setParameter("old", "Create");
update.setParameter("new", "Pay");
if (update.execute() != 1) {
// rollback...
return;
}
}
但是, 在代码中嵌入 SQL, 多少还是挺难受的, 一旦表名或字段名发生变更(尽管这种情况并不多见)就难受了, 虽然可以挪到 xml, 但是在我看来那样更难受.
我的意思是, 能不能在 ebeanServer 中添加一个像下面这样的方法:
public interface EbeanServer {
...
int update(Object newBean, Object oldBean) throws OptimisticLockException;
...
}
这样一来, 调用时, 我只需要
Order order = new Order();
order.setStatus("pay");
Order old = new Order();
old.setId(id);
old.setStatus("Create");
if (ebeanServer.update(order, old) != 1) {
log...
// trigger transaction rollback or do anything with business-related operations
throw new ServiceException("change status fail. please check! order_id: {}", id);
}
当然, 可能带 where 的条件规则会很多, 不仅仅只是这么简单了. 比如
update t_order set status = 'Pay'
where id = xxx and (status = 'Create' or status = 'Confirm')
delete t_system_log where is_delete = 1 and create_time <= '3 month ago'
会有这种需求, 是因为之前我用的 mybatis, 可以用 example 轻松组装出上面的 sql 出来
Ebean-elasticsearch defaults to paging from+size.
However, in the case of deep paging, the efficiency of this method is very low. For example, from=5000,size=10, ES needs to match and sort on each fragment and get 5000*10 valid data, and then return the last 10 data in the result set. In addition to efficiency problems, there is also an unsolvable problem that ES currently supports the maximum skip value of max_result_window, which defaults to 10000, that is, when from+size > When max_result_window, es returns an error.
So when I was in use Ebean, directly use the text () through the es query, is unable to satisfy my business scenario, because after more than 10000, access to the pager maximum page number is not correct, as far as I know the solution, when paging to hundreds of pages, es can't return data, at this time in order to restore the normal use, We can take an emergency dodge by setting max_result_window to 50000, but this solution is only a temporary fix.
So how do I properly page through Ebean using scroll, or something else I'm doing wrong, or is there a better solution
Looking forward to your reply
Bumped mariadb-java-client version to 3.0.7 to see which tests may break (see #2782)
Using setFetchSize(Integer.MIN_VALUE) (introduced by #56) is not allowed according to the documentation of setFetchSize:
It seems that older MariaDB drivers (<3.x.x) accepted Integer.MIN_VALUE for compatibility reasons, however the new major driver version was apparently rewritten from scratch to closely follow the JDBC specs and thus raises an exception when called with Integer.MIN_VALUE.
According to the documentation the proper solution is to use setFetchSize(1).
My current workaround is to stick with the older driver but I would really like to update.
Change the getter methods to use accessor style
While the current documentation is very clear, that EQ (and only EQ) is supported for encrypted properties, I noticed, that IN is currently not supported (which is semantically only a concatenation of multiple EQ queries) So the documentation is right here ;)
I ponder if all exact match queries ("eq","ne","in","notIn") should work for encrypted properties? I see no technical reason, why they should not.
What do you think?
found = Ebean.find(EBasicEncryptClient.class)
.where()
.in("description", "goodbye")
.in("status", EBasicEncryptClient.Status.TWO)
.in("dob", today) // <-- throws a Data Conversion error
.findOne();
should work
When using a fetch and where on a formula joined field (maybe others are also affected) in a query, the joins are build incorrectly. The table alias is wrong, if you have a where clause with 3 path components and a fetch with 2.
See test case with childPerson -> parentPerson -> effectiveBean
The problem is somewhere located here: https://github.com/ebean-orm/ebean/blob/master/ebean-core/src/main/java/io/ebeaninternal/server/query/SqlTreeBuilder.java#L680
"where" generates an extra join which is effectivly added to the root bean und not to the already existing child (from the select/fetch)
The Combination of @NotNull and @DbForeignKey(noConstraint = true) produces the wrong join type in queries.
use case: We import relations between datasets and we only get the primary key of those in this relation. It is not sure if we get those other datasets per imports, but we want to know if there is a relation.
Fix is suggested in PR
#2722 - Migrate to Jakarta JAXB (jakarta.xml.bind <- javax.xml.bind)
#2770 - Add SqlQuery usingTransaction() for consistency and deprecate ExtendedServer API #2772 - Add ebean-annotation to BOM #2766 #2767 - Reference bean created from JSON can invoke lazy loading by default (didn't prior to 12.13.1) #2762 #2765 - NullPointerException while using findCount with forUpdate bug #2733 #2745 - Rename ModuleInfoLoader to EntityClassRegister and _Ebean$ModuleInfo to EbeanEntityRegister
#2778 - Support a naming convention function that can be applied to metrics names for reporting
#2753 #2754 #2755 #2756 #2746 - Bugs findSingleAttributeList projecting a OneToMany property
#2763 - Fix: toJson must use setIntercept if target bean is passed
#2750 - Stateless update populates OneToMany with softdeleted records during serializing
#2748 - Second fetch inside transaction doubles the fetched @OneToMany rows when .setLoadBeanCache(false)
#2742 - ModelContainer.applyChangeSet(ModelContainer.java:167) - IllegalArgumentException: No rule for
#2764 - Fix: DDL will get wrong hint for special migration
#2757 - Add notIn(Query) to query beans #2768 - ENH: Add eqIfPresent() - a helper expression for fluid style when some EQ are optional enhancement #2760 - Add ServerMetrics.asJson() ... to support conversion of collected metrics to json #2749 - Bump ebean-datasource, ebean-annotation, ebean-types, persistence-api #2743 - Postgres DDL - For create table, improve column ordering for tighter storage
#2730 - Secondary @OneToMany fetch query is not happening if the same Entity type is fetched
#2737 - Mapping of TimeZone doesn't have enough characters
#2727 - Stateless update on one-to-many relationship with children soft delete results in a hard delete instead of only inserting new child bug
#2741 - Bump avaje-config to 2.1 and modify ebean-test to support use of ${docker.host} in test container jdbc urls #2734 - maven build - exluding maven descriptor (pom and properties) from built artifacts #2732 - ebean-agent - skip transactional profiling aware enhancement on query beans #2731 - Change ebean-querybean annotations from Retention RUNTIME to CLASS refactor #2723 - Gradle plugin enhancement with Kotlin 1.7.0 bug #2740 - Bump ebean-test-containers to 6.2
#2736 - ENH: findDto nonsupport setter chain (accessor style setters, both fluid style and non fluid style)
#2719 - DDL - Refactor DDL for Postgres and YugabyteDB to just create a default partition table #2718 - [Yugabyte] Fix such that builtin table partition functions are included in DDL (when we use table partitioning) bug #2717 - Support Postgres JSONB/JSON with String - @DbJsonb String content; #2716 - Refactor ScalarType interface to make use of default methods refactor #2715 - [ebean-test-containers] Fix for Postgis to support extraDb (second database), Update Oracle container #2713 - Improve javadoc of Finder, BeanFinder, BeanRepository - deprecate use of BeanFinder server field (migrate to database) #2712 - Bump jedis to 4.2.3 (from 4.2.2) dependencies #2711 - Gradle: Change querybean generator from isolating to aggregating #2710 - JSONB fields are not considered dirty after modified in BeanPersistController preUpdate bug #2708 - Part 1: Fix read/write/query LocalTime values (when time zone configured to be used) #2706 - ENH: Add support for findSingleAttributeSet()
#2704 #2702 - IllegalArgumentException: No rule for io.ebeaninternal.dbmigration.migration.RenameTable #2703 - ebean-test-docker was renamed to ebean-test-containers #2701 - [DDL] Postgres DDL generation for table partitioning with schema - helper method to add partitions does not work bug
#2697 - Postgres DDL - With Postgres table partitioning, add partition column to primary key #2695 - DDL Generation for @UniqueConstraint ignores the name - should be the index name when given #2694 - Postgres DDL - For Postgres table partitioning, remove the "suppressPrimaryKeyOnPartition" mode #2693 - DDL - Add support for creating schema via database migrations (and create all ddl) #2692 - [querybean generation] Compile error, incorrect generation with nested inner class/record @Embeddable and @IdClass
