We are encountering a problem where a small number of messages are not triggering the MessageListener. We have a shared folder where our host drops files to be delivered to all our retail stores. We have a process that has to collect a full set of files as a set and when the set is found, we put each file on a local queue on a centralized server. The MessageListener gets called and we attempt to put the new message on a remote queue for retail store.
Generally this works fairly well, but we have started encountering problems where, according to our logging, we can see a log message that the (file) message was put in the local queue, but in some small percentage of the cases we never get the onMessage() for the MessageListener to put the message on the remote store, which breaks our process.
I’m looking for help/suggestions on how to debug this process to make sure that our MessageListener gets called. Is there a way to request ActiveMQ debug logging without debugging logging that entire application? This is happening in production but not in QA or local dev.
We are running WildFly 16.0.0.Final with ActiveMQ core 5.7.0 and ActiveMQ Artemis 2.6.3.
We have added extra logging around the our executor service and the MessageListener but some files/messages seem to either 1) NOT make it into the local queue or 2) the MessageListener doesn’t fire. Based on the logging it seems like the message gets to the local queue as we found no exceptions around that time-frame so seems to be the listener.
Can you elaborate on your application components? What are you using as a broker? Is it ActiveMQ Artemis 2.6.3 embedded in WildFly 16.0.0.Final? Are your MessageListener implementations running in WildFly (e.g. as MDBs) or elsewhere? How does “ActiveMQ core 5.7.0” fit in and what exactly do you mean by “core” in this context?
These components are pretty old. Both WildFly 16.0.0.Final and ActiveMQ Artemis 2.6.3 were released in early 2019 and ActiveMQ 5.7.0 was released in late 2012. Has there been any thought given to upgrading these components?
How are you sending the messages? Can you paste the code from the producer? Also, what library are you using to send the messages?
Have you gathered any statistics from the queue where you’re sending messages (e.g. Consumer Count, Message Count, Delivering Count, etc.)? If so, what did they show? If not, could you gather these before and after you see the problem?