Add support for transaction bound application events [SPR-12080]

[spring-projects-issues]

Oliver Drotbohm opened SPR-12080 and commented

Spring provides an ApplicationEventPublisher API that can be used to publish events to other application components. By default, the events are published synchronously using a SimpleApplicationEventMulticaster when invoking ApplicationEventPublisher.publishEvent(…) directly.

In user applications the events publish very often signal the outcome of a business action (e.g. "user created"). Thus it's crucial that these events are only published if the transaction succeeds and thus also after the transactions has concluded.

In this repository I built a proof of concept implementation of an ApplicationEventMulticaster that registers a TransactionSynchronization for ApplicationEvents of a certain type (TransactionBoundApplicationEvent in my case). This way, the immediate multicasting is delayed to after the transaction commit.

This basically works and can be seen in action in the Spring RESTBucks example (necessary configuration, the transaction-bound event and event throwing code). There are a few things that could be changed, added on top or improved:

• Instead of the type-based detection of transaction-bound events could be augmented by inspecting the event object for a dedicated annotation.
• Currently the special TransactionAwareApplicationEventMulticaster has to be configured manually currently. However, it could probably registered automatically if the Spring transaction module is on the classpath. (I can also imagine Boot doing that kind of auto-configuration for now).

---

Issue Links:

• CDI-like transaction-aware events for [SPR-10633] #15261 CDI-like transaction-aware events for ("is duplicated by")
• ApplicationListener-like annotation for consuming application events [SPR-11622] #16245 ApplicationListener-like annotation for consuming application events

Referenced from: commits 4741a12

0 votes, 13 watchers

[spring-projects-issues]

Jean-Baptiste Nizet commented

This is something that CDI provides and that is really missing in Spring, IMHO.

Nevertheless, I'd like to suggest another approach for several reasons:

1. Neither the event producer nor the event itself should decide when then observing methods should be notified. Some observers of a given event might want to participate in the transaction, while others might want to be notified of the same event only after commit
2. Being forced to extend a Spring class to implement the event, and to provide a source for the event, is not user-friendly. Providing this (the current bean instance) as the source is even dangerous, IMHO, since the event observer might call back the source and bypass the proxies around the bean
3. Being forced to implement a Spring interface to listen to events is not user-friendly. Especially given that it only allows a component to listen to a single event type. A bean should be able to listen to several event types.

Here's a repo where I implement what I have in mind, and which is quite close to what CDI does. The README explains the design goals and show how to use it. The implementation works fine and should be fast at firing events. I don't know the internals of Spring well enough to know if a bean postprocessor is the best way to support this functionality, though. The implementation depends on Guava, but could easily be modified to have no dependency other than Spring. I just wanted an easy way to have a cache.

https://github.com/Ninja-Squad/spring-events

The project builds and tests pass with Java 6, 7 and 8. More details are available in the javadoc, that can be generated using ./gradlew javadoc.

[spring-projects-issues]

Oliver Drotbohm commented

1. Do you have an example for when it's really necessary to let the observer decide whether it wants to process an even in-transaction or outside it? I really had a hard time coming up with something reasonable. I think usually the point in time you publish the event you really know which approach you need as the opposite one would create inconsistencies or not even reasonable (as the observer might not even see data not yet commited). Your consumer example is duplicating intent here as activating asynchronous event handling effectively requires after-transaction semantics. That looks like an invitation for bugs to some degree.

2. The reason that is, is that the Spring ApplicationEvent publishing mechanism is currently expecting ApplicationEvent being the base class for all events. This is very unlikely to change and switching to completely different strategy is going to make the code accepted highly unlikely. Me using this is more of a testing issue here. Usually the source of the event is the aggregate root that caused the event.

3. Using a dedicated subtype to distinguish between transaction-bound and standard events is just one way and doesn't make it too different from the current way where you have to extend ApplicationEvent. As indicated in the original description other detection mechanisms like an annotation are thinkable. Also, as indicated in 1, I doubt that more flexibility is necessary but I'll change my mind once we come up with a good scenario that requires it.

Generally speaking, the approach I provided didn't have the goal to copy the CDI eventing mechanism but rather extend the current application event mechanism in an idiomatic way. Trying to force a completely different design down Spring's throat is neither going to work nor likely to be accepted.

As indicated in the current model creates dependencies to Spring on the producer and consumer side of things but a mitigating layer could easily be written on top of that. So I think extending the current mechanism with the general capability of hooking into transaction boundaries is a good step. Developers accepting the dependency can the use it as is. Purists trying to avoid it can then build whatever they want on top of that.

[spring-projects-issues]

Jean-Baptiste Nizet commented

First of all, thanks for reading and considering my comment.

1. Yes. In a real, complex application we're building (with CDI, not Spring), each time we ask a customer to reduce its electric consumption, we fire an event which

• causes a billing record to be produced (transactional - failing to produce the billing record must prevent the electric request to be produced)
• causes a monitoring task and alerts to be generated (transactional)
• causes email, voice and text messages to be sent (not transactional and asynchronous, because the web services are not transactional anyway, and because we don't want these long, blocking web service calls to prevent the response to be sent).

To me, the main point of an event mechanism is to have loose coupling between the producer and the consumers of the event. The producer shouldn't know and care about who the consumers are, and how they need to consume the event, IMHO. Regarding my example, sure this asynchronous event requires after-transaction semantics. But the reverse is not always true: you could want an after-transaction event without asynchronous consumption. And as soon as you're using asynchronous methods, yes, you need to think about transaction semantics. This doesn't look more like an invitation to bugs than being able to call an asynchronous method directly from within a transaction.

2. I know and understand this. But this looks, to me, like legacy design, that doesn't match with the annotation-based strategy generally preferred in Spring nowadays. Just like old inheritance-based MVC controllers (MultiActionController, etc.) were replaced by RequestMapping-annotated methods, and PostConstruct-annotated methods are now preferred to implementing InitializingBean. The fact that a component can only listen to one kind of event is really something that bothers me in the current design. Many much more important parts of Spring were completely overhauled in the past to rely on annotations and loose-coupling. Events could follow the same route.

I understand that my proposition doesn't fit into the current event model, but the beauty of Spring is that it's open and flexible enough to allow me to use my code without waiting for official Spring support. I just wanted to provide food for thought. If Spring starts considering improvements to its event model, I'd prefer it to go my suggested way, rather than to patch the current poor (IMO) model.

[spring-projects-issues]

Stéphane Nicoll commented

Annotation-based event listener is on the agenda for 4.2 (see 1bb2a20 and my event branch of my fork). Removing the need for an ApplicationEvent is not on the agenda however and I must say I am still a bit confused as how we could approach this. Jean-Baptiste, this addresses your point 3 but not (yet) point 2.

As for who decides what, I find very disturbing that an event could cause the current transaction to fail; events are processed synchronously by default but you can configure a TaskExecutor to run them asynchronously. Even if it was invoked synchronously, I would find it acceptable that the actual exception thrown by an event listener is not thrown back at me (which is what you use to rollback the current transaction I assume). Decoupling means decoupling, not decoupled components that could rollback a transaction they're not supposed to have access to...

If we take that out for a second (I guess you'll disagree since that's your current design :-) ), then events can be transactional or we don't really care. There are two says to tackle this.

The producer send a "transactional event"

Either with a base class or by adding a @TransactionalEvent (?) on the ApplicationEvent type. Either way, we can configure the phase at which the event should be fired (through admittedly, I haven't found a use case for getting the event only if the transaction rollbacks for instance). This puts the event listener completely in the blind: it receives an event (at the right time) and processes it.

Of course this should only happen if a transaction is actually running.

The receiver request a "transactional event"

We could easily flag a receiver to be transactional if we wanted to TransationalEventListener ?) and register the proper TransactionSynchronization callback when we loop over the listeners. This would require to break SmartApplicationListener probably to give us the extra info. If we put the use case of rollbacking the current transaction aside, I don't really see how that would make sense. The listener should consume and process the event and should not decide "when" to receive it.

Putting your "billing record" use case on the table again, this looks like something you could (should) do without the event infrastructure.

Thoughts?

[spring-projects-issues]

Jean-Baptiste Nizet commented

Being forced to extend a Spring class for the event is not ideal, but it's not the most bothering thing. I can live with that.

To me, not being able to cause a transaction to rollback from a synchronous event would really be a showstopper, though, and I don't think it's viable anyway. In a typical Spring + JPA/Hibernate application, you want your event listener to be able to access the database, and see its state as it is in the current transaction. You're thus de facto in the current transaction. And if you blindly catch exceptions thrown from the listener method (potentially by JPA/Hibernate) as if nothing happened, you'll be in a situation where the main method continues executing, but with a persistence context/hibernate session that is not usable anyway, since JPA/Hibernate exceptions are not recoverable and leave the context/session in an inconsistent state.

Sure, I could implement my billing record use-case without the event infrastructure. But it wouldn't be as elegant, would be tightly coupled, and would introduce hard to remove circular dependencies between modules: the billing module needs to access the operation module to get details about the operation to bill, and the operation module needs to access the billing module to inform it that a new operation must be billed. This is easily and elegantly solved by using synchronous events that participate in the current transaction and allow it to rollback.

Again, I'm not inventing a new wheel here. Synchronous events throwing exceptions and causing the transaction to rollback exist in CDI. And not only that: that's the default behavior. Quote from the spec:

Observer methods may throw exceptions:
If the observer method is a transactional observer method, any exception is caught and logged by the container.
Otherwise, the exception aborts processing of the event. No other observer methods of that event will be called. The BeanManager.fireEvent() or Event.fire() method rethrows the exception. If the exception is a checked exception, it is wrapped and rethrown as an (unchecked) ObserverException.

And again, being able to process the same event in two different ways, and thus to decide that at the listener side is a real usecase to me. Once again, that's how it's done in CDI and that has proven to be useful and practical. See http://docs.oracle.com/javaee/6/api/javax/enterprise/event/Observes.html.

[spring-projects-issues]

Stéphane Nicoll commented

I do know how CDI works :) Again, that's just an opinion and I am happy to open the discussion to a wider group.

[spring-projects-issues]

Stéphane Nicoll commented

Here is a first cut of the implementation (you can look at that branch to check other related commits). It definitely needs some review for corner cases and I want to triple check the potential side effect of using @TransactionalEventListener together with @Transactional in the after completion phase.

Of course, a first look at the current proposal would be great, please let me know if you find anything suspicious.

[spring-projects-issues]

Oliver Drotbohm commented

Awesome, Stéphane, I very much like what I see! Great work! Added a few minor comments to the commit but basically it's a very much improved successor of what I originally imagined and should actually also cover the use cases Jean-Baptiste Nizet outlined, right? Although it will only cover the simple after-commit use case, I'll go ahead and update Spring RESTBucks to use that new stuff in a branch and see how this works out. Will keep you posted!

[spring-projects-issues]

Stéphane Nicoll commented

Thanks Ollie! The code has ben revisited and is available on my branch:
https://github.com/snicoll/spring-framework/commits/event

Jean-Baptiste Nizet would you mind looking at it as well?

[spring-projects-issues]

Jean-Baptiste Nizet commented

Sorry for not having taken the time to review this sooner.

I won't comment on the internals of the implementation, because you're much more qualified than me on this. But this looks great regarding the end-user API. So, to recap, please correct me if I'm wrong:

• an event can be any class, or event a generic type (i.e. Foo<Bar> can be treated differentiated from Foo<Baz>)
• you can annotate any method with a single argument of a Spring bean with @EventListener. In this case the method is called when the event is published, the return type is treated as a new event, and any exception is wrapped into a runtime exception and thrown. The method can also be annotated with Async for the event to be handled asynchronously
• for more transactional semantics, any such method can be annotated with @TransactionalEventListener. The same rules apply, except the listener is called, immediately, before commit, after commit or after rollback depending on the phase attribute of the annotation.
• this annotation can also be used to define meta-annotations
• you publish an event by calling a method of the Spring application context.

If all that is correct, it all looks great to me, except I would find it cleaner if there was an interface, implemented by an injectable bean, that could be used to publish events. I could of course define such an interface and such a bean myself, so that's not a big problem:

@Autowired
private EventPublisher eventPublisher;

public void someServiceMethod() {
    eventPublisher.publish(new UserRegistered(userId);
}

[spring-projects-issues]

Stéphane Nicoll commented

That's ok. There's no immediate phase for @TransactionalEventListener, just use @EventListener if you want to listen to the event immediately.

There is an interface to publish an event and it already exists (it was just extended a bit). just inject ApplicationEventPublisher in your code and you'll get that publish method. The context implements the interface so it's always available .

[spring-projects-issues]

Oliver Drotbohm commented

I just created an example for this and wondered why we have BEFORE_COMMIT but no BEFORE_ROLLBACK?

[spring-projects-issues]

Stéphane Nicoll commented

No reason. Except this: why would you use that? You know the current transaction is going to rollback...

[spring-projects-issues]

Oliver Drotbohm commented

I saw there's not even a method in TransactionSynchronization to support this. I was just puzzled by asymmetry. I can imagine notifying someone about the fact that an event has been triggered but the event publishing code rolling back. Just wanted to make sure it's not an oversight.