Skip to content

NullPointerException during JPA deleteAll with lazy collections #35617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pwandl opened this issue May 24, 2023 · 12 comments
Closed

NullPointerException during JPA deleteAll with lazy collections #35617

pwandl opened this issue May 24, 2023 · 12 comments
Assignees
Labels
for: external-project For an external project and not something we can fix status: invalid An issue that we don't feel is valid

Comments

@pwandl
Copy link

pwandl commented May 24, 2023

Since upgrading to Spring Boot 3.1.0 from 3.0.6 we are experiencing NPEs when calling .deleteAll() on Spring Data JPA repositories, but not for each entity. I suspect it is related to entities which contain lazy collections, but could not narrow it down further yet.

I have extracted a minimal sample application which shows the issue: https://github.com/pwandl/spring-hibernate-bug-demo

It seems to only occur if the transaction calling .deleteAll has not previously modified the to-be-deleted entities. I was not yet able to reproduce the issue using hibernate directly without any spring components, so I suspect that it might be related to some spring component, even though the issue does not occur when downgrading hibernate to 6.1.7

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label May 24, 2023
@wilkinsona wilkinsona self-assigned this May 24, 2023
@wilkinsona
Copy link
Member

Thanks for the sample. While I haven't managed to reproduce the problem without any usage of Spring, I have reproduced it without Spring Boot and Spring Data JPA. I've pushed a commit to my fork of your sample that just uses Spring Framework and Hibernate. It should fail with the same NPE.

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
May 24, 2023 5:25:06 PM org.hibernate.jpa.internal.util.LogHelper logPersistenceUnitInformation
INFO: HHH000204: Processing PersistenceUnitInfo [name: default]
May 24, 2023 5:25:06 PM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate ORM core version 6.2.2.Final
May 24, 2023 5:25:06 PM org.hibernate.cfg.Environment <clinit>
INFO: HHH000406: Using bytecode reflection optimizer
May 24, 2023 5:25:06 PM org.hibernate.bytecode.internal.BytecodeProviderInitiator buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : bytebuddy
May 24, 2023 5:25:07 PM org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl logSelectedDialect
INFO: HHH035001: Using dialect: org.hibernate.dialect.H2Dialect, version: 2.1.214
May 24, 2023 5:25:07 PM org.hibernate.bytecode.internal.BytecodeProviderInitiator buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : bytebuddy
May 24, 2023 5:25:07 PM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 8.0.0.Final
May 24, 2023 5:25:08 PM org.hibernate.engine.transaction.jta.platform.internal.JtaPlatformInitiator initiateService
INFO: HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
Hibernate: create table application_user (id bigint not null, createdBy_id bigint, primary key (id))
Hibernate: create table User_userRoles (User_id bigint not null, userRoles varchar(255) check (userRoles in ('USER','ADMIN','OPERATOR')))
Hibernate: create sequence application_user_SEQ start with 1 increment by 50
Hibernate: alter table if exists application_user add constraint FKiag4pd92c0slpoao47m5i793b foreign key (createdBy_id) references application_user
Hibernate: alter table if exists User_userRoles add constraint FKbuid0qjp6u03wg7qj2fnpf4e8 foreign key (User_id) references application_user
Hibernate: select next value for application_user_SEQ
Hibernate: select next value for application_user_SEQ
1
Hibernate: insert into application_user (createdBy_id,id) values (?,?)
Hibernate: insert into application_user (createdBy_id,id) values (?,?)
Hibernate: insert into User_userRoles (User_id,userRoles) values (?,?)
Hibernate: insert into User_userRoles (User_id,userRoles) values (?,?)
Hibernate: select u1_0.id,u1_0.createdBy_id from application_user u1_0
Hibernate: delete from User_userRoles where User_id=?
Hibernate: delete from User_userRoles where User_id=?
Exception in thread "main" org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:570)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:152)
	at com.example.demo.DemoApplication.deleteUsers(DemoApplication.java:47)
	at com.example.demo.DemoApplication.main(DemoApplication.java:41)
Caused by: jakarta.persistence.RollbackException: Error while committing the transaction
	at org.hibernate.internal.ExceptionConverterImpl.convertCommitException(ExceptionConverterImpl.java:65)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:561)
	... 5 more
Caused by: jakarta.validation.ValidationException: HV000028: Unexpected exception during isValid call.
	at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:186)
	at org.hibernate.validator.internal.engine.constraintvalidation.SimpleConstraintTree.validateConstraints(SimpleConstraintTree.java:66)
	at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateConstraints(ConstraintTree.java:75)
	at org.hibernate.validator.internal.metadata.core.MetaConstraint.doValidateConstraint(MetaConstraint.java:130)
	at org.hibernate.validator.internal.metadata.core.MetaConstraint.validateConstraint(MetaConstraint.java:123)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateMetaConstraint(ValidatorImpl.java:555)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForSingleDefaultGroupElement(ValidatorImpl.java:518)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForDefaultGroup(ValidatorImpl.java:488)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForCurrentGroup(ValidatorImpl.java:450)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateInContext(ValidatorImpl.java:400)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validate(ValidatorImpl.java:172)
	at org.hibernate.boot.beanvalidation.BeanValidationEventListener.validate(BeanValidationEventListener.java:128)
	at org.hibernate.boot.beanvalidation.BeanValidationEventListener.onPreUpdate(BeanValidationEventListener.java:92)
	at org.hibernate.action.internal.EntityUpdateAction.preUpdate(EntityUpdateAction.java:329)
	at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:159)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
	at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:489)
	at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:486)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:358)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1412)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:485)
	at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2296)
	at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:1961)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:439)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:169)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:267)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
	... 6 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.hibernate.persister.collection.CollectionPersister.isExtraLazy()" because "persister" is null
	at org.hibernate.collection.spi.AbstractPersistentCollection.lambda$readSize$0(AbstractPersistentCollection.java:155)
	at org.hibernate.collection.spi.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:265)
	at org.hibernate.collection.spi.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:148)
	at org.hibernate.collection.spi.PersistentSet.size(PersistentSet.java:151)
	at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:37)
	at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:22)
	at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:180)
	... 35 more

Given that we can't (yet?) reproduce the problem without Spring Framework, I think we should get the Framework team to take a look. To that end, please open a Spring Framework issue referencing this issue and my updated version of your sample.

@wilkinsona wilkinsona closed this as not planned Won't fix, can't repro, duplicate, stale May 24, 2023
@wilkinsona wilkinsona added status: invalid An issue that we don't feel is valid for: external-project For an external project and not something we can fix and removed status: waiting-for-triage An issue we've not yet triaged labels May 24, 2023
@quaff
Copy link
Contributor

quaff commented May 25, 2023

I think it's bug of hibernate 6.2, It works fine when downgrade to 6.1.7, you should report to hibernate team not spring team.

@wilkinsona
Copy link
Member

@pwandl already said that in the opening comment:

I was not yet able to reproduce the issue using hibernate directly without any spring components, so I suspect that it might be related to some spring component, even though the issue does not occur when downgrading hibernate to 6.1.7

However, thus far we've been unable to reproduce the problem without Spring. If you have managed to do so, @quaff, please do share the code.

@quaff
Copy link
Contributor

quaff commented May 25, 2023

I would take a look if @pwandl share the code that works fine using hibernate directly without any spring components.

@jhoeller
Copy link

jhoeller commented May 25, 2023

As far as I was able to reproduce this, neither Spring transaction management nor HibernateJpaVendorAdapter have any impact here since the problem remains even with plain EntityManagerFactory.createEntityManager / EntityManager.getTransaction().begin() etc and with HibernatePersistenceProvider configuration on our LocalContainerEntityManagerFactoryBean, coming close to plain Hibernate usage in practice.

The one thing that makes it pass is removing the Jakarta EL provider org.apache.tomcat.embed:tomcat-embed-el:10.1.8 from the classpath. Replacing it with org.glassfish:jakarta.el:4.0.2 breaks as well, whereas it still passes with the presence of the plain EL API in the form of jakarta.el:jakarta.el-api:4.0.0.

So it looks like something is reacting to the presence of a Jakarta EL provider here, registering some special handling that ultimately breaks persister availability for lazy collection introspection. I have a suspicion that this is related to Hibernate Validator but it might be in Hibernate ORM 6.2 itself since a plain downgrade to ORM 6.1.7 makes the problem go away.

In any case, this is definitely a Hibernate interaction problem. If someone has the cycles for it, please try to reproduce it with plain Hibernate (bringing a Jakarta EL provider onto the classpath) and report it to the Hibernate project.

@wilkinsona
Copy link
Member

So it looks like something is reacting to the presence of a Jakarta EL provider here, registering some special handling that ultimately breaks persister availability for lazy collection introspection

That matches what I saw too. It's size-related validation that calls a method on AbstractPersistentCollection which breaks. Without an EL provider, Bean Validation backs off so the problem doesn't occur.

@jhoeller
Copy link

I originally could not reproduce this in the form of a test in spring-orm. It took a while to find the reason: namely that there was no EL provider on the classpath there, whereas it was declared in the standalone repro build. This difference unfortunately was not obvious at all.

You got a point there that without an EL provider, Hibernate ORM seems to back off from applying Hibernate Validator completely. A standalone Hibernate Validator setup fails with a proper exception if there is no EL provider on the classpath; I suppose Hibernate ORM silently swallows that and simply does not validate then.

In any case, I expect the AbstractPersistentCollection problem to be reproducible with plain Hibernate as long as there is a complete Hibernate Validator setup with an EL provider on the classpath. The CollectionPersister is meant to be present during flush but is somehow missing in action here, with nothing that we can do about it from the outside.

@jhoeller
Copy link

P.S.: The early failure for an incomplete Hibernate Validator setup can be enforced via setValidationMode(ValidationMode.CALLBACK) on the EMF setup. I wonder whether we should make that the default for Spring-driven setups? On the other hand, what I really want is AUTO with different rules: back off if no validation provider is present but fail early in case of an incomplete validation setup.

@wilkinsona
Copy link
Member

My attempt to reproduce the problem without Framework is here. The debugger shows org.hibernate.boot.beanvalidation.BeanValidationEventListener being called but no failure occurs. Perhaps someone who's more familiar with how Framework configures Hibernate can spot what's missing.

@jhoeller
Copy link

jhoeller commented May 25, 2023

That almost hits the spot, it just shouldn't reuse the EntityManager instance between the transactions but rather create a fresh EntityManager per transaction. The following change to the pure JPA version makes the same exception appear for me:

public class DemoApplication {

	private final EntityManagerFactory entityManagerFactory;

	public DemoApplication() {
		entityManagerFactory = Persistence.createEntityManagerFactory("test");
	}

	public static void main(String[] args) {
		DemoApplication demoApplication = new DemoApplication();
		demoApplication.createUsers();
		demoApplication.deleteUsers();
	}

	private void deleteUsers() {
		EntityManager entityManager = entityManagerFactory.createEntityManager();
		entityManager.getTransaction().begin();
		for (User user : entityManager.createQuery("Select u from User u", User.class).getResultList()) {
			User toDelete = entityManager.find(User.class, user.getId());
			entityManager.remove(entityManager.contains(toDelete) ? toDelete : entityManager.merge(toDelete));
		}
		entityManager.getTransaction().commit();
		entityManager.close();
	}

	private void createUsers() {
		EntityManager entityManager = entityManagerFactory.createEntityManager();
		entityManager.getTransaction().begin();
		User a = new User();
		a.setUserRoles(EnumSet.of(UserRole.USER));
		entityManager.persist(a);
		User user = new User();
		user.setUserRoles(EnumSet.of(UserRole.USER));
		user.setCreatedBy(a);
		entityManager.persist(user);
		entityManager.getTransaction().commit();
		entityManager.close();
		System.out.println(user.getCreatedBy().getId());
	}
}

@wilkinsona
Copy link
Member

Thanks, @jhoeller.

@pwandl I've updated my fork of your example and it now reproduces the problem with pure JPA/Hibernate in the pure-jpa branch:

May 25, 2023 5:47:45 PM org.hibernate.jpa.internal.util.LogHelper logPersistenceUnitInformation
INFO: HHH000204: Processing PersistenceUnitInfo [name: test]
May 25, 2023 5:47:46 PM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate ORM core version 6.2.2.Final
May 25, 2023 5:47:46 PM org.hibernate.cfg.Environment <clinit>
INFO: HHH000406: Using bytecode reflection optimizer
May 25, 2023 5:47:46 PM org.hibernate.engine.jdbc.connections.internal.ConnectionProviderInitiator initiateService
INFO: HHH000130: Instantiating explicit connection provider: org.hibernate.hikaricp.internal.HikariCPConnectionProvider
[main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
[main] INFO com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection conn0: url=jdbc:h2:mem:test user=ROOT
[main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed.
May 25, 2023 5:47:46 PM org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl logSelectedDialect
INFO: HHH035001: Using dialect: org.hibernate.dialect.H2Dialect, version: 2.1.214
May 25, 2023 5:47:46 PM org.hibernate.bytecode.internal.BytecodeProviderInitiator buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : bytebuddy
May 25, 2023 5:47:46 PM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 8.0.0.Final
May 25, 2023 5:47:47 PM org.hibernate.engine.transaction.jta.platform.internal.JtaPlatformInitiator initiateService
INFO: HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
Exception in thread "main" jakarta.persistence.RollbackException: Error while committing the transaction
	at org.hibernate.internal.ExceptionConverterImpl.convertCommitException(ExceptionConverterImpl.java:65)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
	at com.example.demo.DemoApplication.deleteUsers(DemoApplication.java:30)
	at com.example.demo.DemoApplication.main(DemoApplication.java:20)
Caused by: jakarta.validation.ValidationException: HV000028: Unexpected exception during isValid call.
	at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:186)
	at org.hibernate.validator.internal.engine.constraintvalidation.SimpleConstraintTree.validateConstraints(SimpleConstraintTree.java:66)
	at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateConstraints(ConstraintTree.java:75)
	at org.hibernate.validator.internal.metadata.core.MetaConstraint.doValidateConstraint(MetaConstraint.java:130)
	at org.hibernate.validator.internal.metadata.core.MetaConstraint.validateConstraint(MetaConstraint.java:123)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateMetaConstraint(ValidatorImpl.java:555)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForSingleDefaultGroupElement(ValidatorImpl.java:518)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForDefaultGroup(ValidatorImpl.java:488)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForCurrentGroup(ValidatorImpl.java:450)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateInContext(ValidatorImpl.java:400)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validate(ValidatorImpl.java:172)
	at org.hibernate.boot.beanvalidation.BeanValidationEventListener.validate(BeanValidationEventListener.java:128)
	at org.hibernate.boot.beanvalidation.BeanValidationEventListener.onPreUpdate(BeanValidationEventListener.java:92)
	at org.hibernate.action.internal.EntityUpdateAction.preUpdate(EntityUpdateAction.java:329)
	at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:159)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
	at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:489)
	at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:486)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:358)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1412)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:485)
	at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2296)
	at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:1961)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:439)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:169)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:267)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
	... 2 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.hibernate.persister.collection.CollectionPersister.isExtraLazy()" because "persister" is null
	at org.hibernate.collection.spi.AbstractPersistentCollection.lambda$readSize$0(AbstractPersistentCollection.java:155)
	at org.hibernate.collection.spi.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:265)
	at org.hibernate.collection.spi.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:148)
	at org.hibernate.collection.spi.PersistentSet.size(PersistentSet.java:151)
	at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:37)
	at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:22)
	at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:180)
	... 31 more

Please report this to the Hibernate team.

@pwandl
Copy link
Author

pwandl commented May 26, 2023

Thanks for the detailled analysis!
Hibernate issue can be found here: https://hibernate.atlassian.net/browse/HHH-16701

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
for: external-project For an external project and not something we can fix status: invalid An issue that we don't feel is valid
Projects
None yet
Development

No branches or pull requests

5 participants