org.infinispan.interceptors.InvocationContextInterceptor: ... Не удалось получить блокировку на

Question

Мы используем infinispan 5.3.0 в многосерверном кластере для приложения Glassfish (3.1.2). В приложении есть пользователи, обрабатывающие события, которые хранятся в бесконечном кэше.

При определенных (?) Пик раз мы видим много следующих stacktraces в infinispan журналах:

2015.04.24_14:41:47,482 - ERROR org.infinispan.interceptors.InvocationContextInterceptor: ISPN000136: Execution error [Thread=http-thread-pool-26861(93)] 
org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on 66f2ebb9-cb22-4ba1-be6f-55ca0167cab6 on behalf of transaction GlobalTransaction:<[servername]-30723>:283157:local. Lock is being held by GlobalTransaction:<[servername]-59526>:141148:remote 
     at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.newTimeoutException(AbstractTxLockingInterceptor.java:238) 
     at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.waitForTransactionsToComplete(AbstractTxLockingInterceptor.java:231) 
     at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:191) 
     at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:136) 
     at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:255) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) 
     at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134) 
     at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:169) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) 
     at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:130) 
     at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:189) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) 
     at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134) 
     at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:169) 
     at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:97) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) 
     at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216) 
     at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:189) 
     at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:131) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) 
     at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134) 
     at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:169) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) 
     at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128) 
     at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:97) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343) 
     at org.infinispan.CacheImpl.lock(CacheImpl.java:672) 
     at org.infinispan.DecoratedCache.lock(DecoratedCache.java:139) 


2015.04.24_14:46:39,010 - ERROR org.infinispan.remoting.InboundInvocationHandlerImpl: Exception executing command [Thread=remote-thread-13] 
org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on [event-lock-id>] on behalf of transaction GlobalTransaction:<[servername]-37484>:26400:remote. Lock is being held by null 
     at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.newTimeoutException(AbstractTxLockingInterceptor.java:238) 
     at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.waitForTransactionsToComplete(AbstractTxLockingInterceptor.java:231) 
     at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:191) 
     at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:136) 
     at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:255) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) 
     at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134) 
     at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:169) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) 
     at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:130) 
     at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:189) 
     at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:132) 
     at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) 
     at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134) 
     at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:169) 
     at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:94) 

Имеет anbody идею, что причиной этих ошибок или каких-либо указателей для дальнейшего анализа?

Заранее благодарен!

Answer 1

Infinispan использует распределенные блокировки для синхронизации модификации кеша. Причиной этого является то же, что и синхронизация многопоточного доступа к переменной. Во время пикового времени блокировка конкуренции становится слишком высокой, и некоторые попытки блокировки блокируются. Рассмотрим увеличение тайм-аута блокировки или настройку блокировки. Подробную информацию см. В Руководстве пользователя Infinispan: 5. Locking and Concurrency.