Reconciliation stopped working

Dear community,

After a planned server downtime (where I have OpenIAM installed), and a missed run of one reconciliation job due to the downtime, the reconciliation process has completely stopped working. The log shows the following error (separate message).

Could you please advise how to resolve this issue?

Thank you very much in advance.
Jan

org.springframework.batch.core.repository.JobExecutionAlreadyRunningException: Previous reconciliation run is not finished yet at org.openiam.recon.service.impl.service.ReconciliationDataServiceImpl.preStartChecks(ReconciliationDataServiceImpl.java:191) at org.openiam.recon.service.impl.service.ReconciliationDataServiceImpl.startAsynchReconciliation(ReconciliationDataServiceImpl.java:78) at org.openiam.recon.mq.AsynchronReconciliationMessageListener$2.doProcess(AsynchronReconciliationMessageListener.java:69) at org.openiam.recon.mq.AsynchronReconciliationMessageListener$2.doProcess(AsynchronReconciliationMessageListener.java:61) at org.openiam.common.mq.listener.AbstractListener.processRequest(AbstractListener.java:125) at org.openiam.common.mq.listener.AbstractListener.processingCrudRequest(AbstractListener.java:311) at jdk.internal.reflect.GeneratedMethodAccessor87.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:169) at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:119) at org.springframework.amqp.rabbit.listener.adapter.DelegatingInvocableHandler.invoke(DelegatingInvocableHandler.java:186) at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:88) at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:261) at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandlerAndProcessResult(MessagingMessageListenerAdapter.java:207) at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:146) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1665) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1584) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1572) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1563) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1507) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:967) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:914) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:83) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1291) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1197) at java.base/java.lang.Thread.run(Unknown Source) BasicDataServiceException [code=RECON_CONFIG_ALREADY_RUNNING, responseValue=null, originalCause=null, errorTokenList=null] at org.openiam.recon.service.impl.service.ReconciliationDataServiceImpl.preStartChecks(ReconciliationDataServiceImpl.java:193) at org.openiam.recon.service.impl.service.ReconciliationDataServiceImpl.startAsynchReconciliation(ReconciliationDataServiceImpl.java:78) at org.openiam.recon.mq.AsynchronReconciliationMessageListener$2.doProcess(AsynchronReconciliationMessageListener.java:69) at org.openiam.recon.mq.AsynchronReconciliationMessageListener$2.doProcess(AsynchronReconciliationMessageListener.java:61) at org.openiam.common.mq.listener.AbstractListener.processRequest(AbstractListener.java:125) at org.openiam.common.mq.listener.AbstractListener.processingCrudRequest(AbstractListener.java:311) at jdk.internal.reflect.GeneratedMethodAccessor87.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:169) at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:119) at org.springframework.amqp.rabbit.listener.adapter.DelegatingInvocableHandler.invoke(DelegatingInvocableHandler.java:186) at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:88) at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:261) at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandlerAndProcessResult(MessagingMessageListenerAdapter.java:207) at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:146) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1665) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1584) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1572) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1563) at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1507) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:967) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:914) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:83) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1291) at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1197) at java.base/java.lang.Thread.run(Unknown Source)

The problem still persists. Does anyone know how to solve this, please?

Hello @Bizz,

Please try the following:

  1. Stop OpenIAM
  2. Clear the Redis cache by running the following commands in Redis-CLI
FLUSHDB

And then:

FLUSHALL

Please let us know if this resolves the issue.

Thanks,
Ameet

Hello Ameet,

it doesn’t work. Still the same error.

Jan

Hello Jan,

Could you share the output from the Redis commands above? Also, could you share which version of OpenIAM you are running?

Thanks.

Hello Ameet,

Redis output here:

Version is 4.2.1.13

Thanks

Jan

Hello Jan,

Could you please upgrade to version 4.2.1.14? We have documentation for upgrading at this link.

This is the latest version of the Community Edition, and should help resolve the issues your facing.

Thanks,

Ameet