I am currently utilizing Spring Boot 2.7.18 alongside the MongoDB sync driver. Within my codebase, there exists a transactional method called “updateCertCacheCollectionOld.” This method is responsible for removing approximately 7000 outdated records and subsequently inserting an equivalent number of new records. Ordinarily, the execution of this method takes only a few seconds to complete. However, today an exception was thrown, specifically a com.mongodb.MongoCommandException with an error code of 251 (NoSuchTransaction).
The version of MongoDB being employed is 5.x with replicateSet.
Does anyone have idea about the expceiton
@Transactional(propagation = Propagation.REQUIRED)
public void updateCertCacheCollectionOld(String backendCode, List<CertCache> backendCache) {
Instant start = Instant.now();
logger.info("updateCertCacheCollection[{}], receive {}", backendCode, backendCache.size());
List<String> prefix = configSvc.getBackend(backendCode).getPrefixs().stream().map(o -> o.getId()).collect(Collectors.toList());
long deleteCount = certCacheDao.removeCertPrefixWith(prefix);
logger.info("updateCertCacheCollection[{}], delete {} records with prefix {}", backendCode, deleteCount, prefix);
long bulkWriteResult = certCacheDao.bulkWrite(insertCert);
logger.info("{}:Bulk insert of {} documents. updateCertCacheCollection completed in {} milliseconds", backendCode, bulkWriteResult,
Duration.between(start, Instant.now()).toMillis());
}
@Override
public long removeCertPrefixWith(List<String> prefixs) {
if (prefixs.size() == 0)
return 0;
Query query = new Query();
List<Criteria> criteria = new ArrayList<>();
prefixs.forEach((prefix) -> {
criteria.add(Criteria.where("data.utn").regex("^" + prefix + ".*"));
});
query.addCriteria(new Criteria().orOperator(criteria.toArray(new Criteria[criteria.size()])));
return mongoTemplate.remove(query, CertCache.class).getDeletedCount();
}
@Override
public long bulkWrite(List<CertCache> list) {
BulkOperations bulkInsertion = mongoTemplate.bulkOps(BulkOperations.BulkMode.UNORDERED, CertCache.class);
for (int i = 0; i < list.size(); ++i) {
bulkInsertion.insert(list.get(i));
}
BulkWriteResult bulkWriteResult = bulkInsertion.execute();
return bulkWriteResult.getInsertedCount();
}
The expceiton log
<43>Jan 19 00:04:37 abcpojobp1.md.hksarg ERROR abcpoJob-prod-1 [scheduling-1] org.mardep.abcpo.service.impl.CronJobService:109 - errors when updateCertCacheCollection[SECS]
org.springframework.transaction.TransactionSystemException: Could not commit Mongo transaction for session [ClientSessionImpl@4231b1c9 id = {"id": {"$binary": {"base64": "8/+013KSTDylmVzuSSRwPA==", "subType": "04"}}}, causallyConsistent = true, txActive = false, txNumber = 12595, closed = false, clusterTime = {"clusterTime": {"$timestamp": {"t": 1705593877, "i": 1}}, "signature": {"hash": {"$binary": {"base64": "KdYLcPKoILyw+HMRLMOP3P2hmuA=", "subType": "00"}}, "keyId": 7281219174569869313}}].; nested exception is com.mongodb.MongoCommandException: Command failed with error 251 (NoSuchTransaction): 'Transaction 12595 has been aborted.' on server abcpodbsp2.md.hksarg:27017. The full response is {"errorLabels": ["TransientTransactionError"], "ok": 0.0, "errmsg": "Transaction 12595 has been aborted.", "code": 251, "codeName": "NoSuchTransaction", "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1705593877, "i": 1}}, "signature": {"hash": {"$binary": {"base64": "KdYLcPKoILyw+HMRLMOP3P2hmuA=", "subType": "00"}}, "keyId": 7281219174569869313}}, "operationTime": {"$timestamp": {"t": 1705593877, "i": 1}}}
at org.springframework.data.mongodb.MongoTransactionManager.doCommit(MongoTransactionManager.java:203)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707)
at org.mardep.abcpo.service.impl.CertCacheUpdateCronJobService$$EnhancerBySpringCGLIB$$53fad461.updateCertCacheCollection(<generated>)
at org.mardep.abcpo.service.impl.CronJobService.lambda$updateCertCache$2(CronJobService.java:106)
at java.base/java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4706)
at org.mardep.abcpo.service.impl.CronJobService.updateCertCache(CronJobService.java:93)
at jdk.internal.reflect.GeneratedMethodAccessor5540.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:95)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.mongodb.MongoCommandException: Command failed with error 251 (NoSuchTransaction): 'Transaction 12595 has been aborted.' on server abcpodbsp2.md.hksarg:27017. The full response is {"errorLabels": ["TransientTransactionError"], "ok": 0.0, "errmsg": "Transaction 12595 has been aborted.", "code": 251, "codeName": "NoSuchTransaction", "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1705593877, "i": 1}}, "signature": {"hash": {"$binary": {"base64": "KdYLcPKoILyw+HMRLMOP3P2hmuA=", "subType": "00"}}, "keyId": 7281219174569869313}}, "operationTime": {"$timestamp": {"t": 1705593877, "i": 1}}}
at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:198)
at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:413)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:337)
at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:116)
at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:644)
at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:71)
at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:240)
at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:226)
at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:126)
at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:116)
at com.mongodb.internal.connection.DefaultServer$OperationCountTrackingConnection.command(DefaultServer.java:345)
at com.mongodb.internal.operation.CommandOperationHelper.lambda$executeRetryableWrite$15(CommandOperationHelper.java:413)
at com.mongodb.internal.operation.OperationHelper.lambda$withSourceAndConnection$2(OperationHelper.java:575)
at com.mongodb.internal.operation.OperationHelper.withSuppliedResource(OperationHelper.java:600)
at com.mongodb.internal.operation.OperationHelper.lambda$withSourceAndConnection$3(OperationHelper.java:574)
at com.mongodb.internal.operation.OperationHelper.withSuppliedResource(OperationHelper.java:600)
at com.mongodb.internal.operation.OperationHelper.withSourceAndConnection(OperationHelper.java:573)
at com.mongodb.internal.operation.CommandOperationHelper.lambda$executeRetryableWrite$16(CommandOperationHelper.java:397)
at com.mongodb.internal.async.function.RetryingSyncSupplier.get(RetryingSyncSupplier.java:65)
at com.mongodb.internal.operation.CommandOperationHelper.executeRetryableWrite(CommandOperationHelper.java:425)
at com.mongodb.internal.operation.TransactionOperation.execute(TransactionOperation.java:70)
at com.mongodb.internal.operation.CommitTransactionOperation.execute(CommitTransactionOperation.java:133)
at com.mongodb.internal.operation.CommitTransactionOperation.execute(CommitTransactionOperation.java:54)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:212)
at com.mongodb.client.internal.ClientSessionImpl.commitTransaction(ClientSessionImpl.java:147)
at org.springframework.data.mongodb.MongoTransactionManager$MongoTransactionObject.commitTransaction(MongoTransactionManager.java:469)
at org.springframework.data.mongodb.MongoTransactionManager.doCommit(MongoTransactionManager.java:236)
at org.springframework.data.mongodb.MongoTransactionManager.doCommit(MongoTransactionManager.java:200)
... 24 common frames omitted
Collapse
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141573, serverValue:26290}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141572, serverValue:26289}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141571, serverValue:26288}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141570, serverValue:26287}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141569, serverValue:26286}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141568, serverValue:26285}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141567, serverValue:26284}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141566, serverValue:26283}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141565, serverValue:26282}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-4-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141564, serverValue:26281}] to abcpodbsp3.md.hksarg:27017
<46>Jan 19 00:04:36 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [scheduling-1] org.mardep.abcpo.service.impl.CertCacheUpdateCronJobService:82 - SECS:Bulk insert of 7577 documents. updateCertCacheCollection completed in 62223 milliseconds
<46>Jan 19 00:04:23 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [MaintenanceTimer-3-thread-1] org.mongodb.driver.connection:71 - Opened connection [connectionId{localValue:141563, serverValue:1262952}] to abcpodbsp2.md.hksarg:27017
<46>Jan 19 00:03:56 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [scheduling-1] org.mardep.abcpo.service.impl.CertCacheUpdateCronJobService:66 - updateCertCacheCollection[SECS], delete 7577 records with prefix [SE]
<46>Jan 19 00:03:34 abcpojobp1.md.hksarg INFO abcpoJob-prod-1 [scheduling-1] org.mardep.abcpo.service.impl.CertCacheUpdateCronJobService:63 - updateCertCacheCollection[SECS], receive 7577
the mongo db config
@Bean
MongoTemplate mongoTemplate(){
MongoTemplate mongoTemplate = new MongoTemplate(mongoClient(), getDatabaseName());
mongoTemplate.setSessionSynchronization(SessionSynchronization.ALWAYS);
return mongoTemplate;
}
@SneakyThrows
@NotNull
@Override
@Bean
public MongoClient mongoClient() {
if (connectionurl.length()!=0) {
return MongoClients.create(connectionurl);
}
String url = "mongodb://" + username + ":" + cryptoSvc.decrypt(encryptedPassword) + "@" + host + "/" + dbname + params;
return MongoClients.create(url);
}
@Bean
MongoTransactionManager transactionManager(MongoDatabaseFactory dbFactory) {
TransactionOptions options = TransactionOptions.builder().writeConcern(WriteConcern.ACKNOWLEDGED).build();
return new MongoTransactionManager(dbFactory, options);
}
The application.properties related to mongoDB
spring.mongodb.host=abcpodbsd2.md.hksarg:27017,abcpodbsd3.md.hksarg:27017,abcpodbsd1.md.hksarg:27017
spring.mongodb.dbname=abcpoDEV
spring.mongodb.params=?replicaSet=replicaset01&minPoolSize=10&maxIdleTimeMS=600000
I have a suspicion that the transaction associated with the underlying connection was closed. In order to prevent the connection from closing, I tried setting maxIdleTimeMS to 0. Additionally, I deliberately added a sleep in the method to simulate a longer transaction. If the sleep duration is relatively short, around 60 seconds, everything operates as expected. However, when the sleep duration is extended, for example, to 300 seconds, the same exception is thrown.
try {
logger.info("sleeping");
Thread.sleep(1000*300);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
logger.info("sleep completed, begin blukwrite");
long bulkWriteResult = certCacheDao.bulkWrite(insertCert);
Does this answer your question? c# How to extend time of mongo db transaction lifetime
see only first option in the above question