I’m using htmlunit with Java, and I’m trying to click a span on a website that should increase a counter and unlock another button, but unfortunately, it is not working.
The warning states, that the element is not displayed, but when I open the website in my browser the element is there and can be clicked.
// Find all services to select "Sonstiges" service
List<DomElement> services = secondPage.getElementsByName("services");
for (DomElement service : services) {
// Get correct service
if (service.getNextElementSibling().getTextContent().equals("Sonstiges")) {
service.getNextElementSibling().getNextElementSibling().getNextElementSibling()
.getNextElementSibling().getChildElements().forEach(domElement -> {
if (domElement.getTextContent().equals("+")) {
// Press add service button
try {
domElement.click();
} catch (IOException e) {
logger.log(Level.WARNING, "Could not add service:\n" + Util.exceptionToString(e));
}
}
});
break;
}
}
// Click submit
DomElement secondSubmitButton = secondPage.getElementById("forward-service");
HtmlPage thirdPage = secondSubmitButton.click();
When I run this code, I get a lot of warnings regarding cookies and CSS, but I am not sure if they are responsible for the last warning, which states that the span with the ‘+’ as text was not clicked, so the secondSubmitButton is not present.
Nov. 06, 2023 3:56:17 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNUNG: Cookie rejected [rc_sid99f7e7c958a84cb084fd362b95200aaa="48f8b876f10b4be18c670eb8fb37998d7b3898197b76443bab04b27a7fae1e9f", version:0, domain:.callup.kaiserslautern.de, path:/m/Fuehrerscheinstelle, expiry:Mon Nov 06 16:16:13 CET 2023] Illegal 'domain' attribute ".callup.kaiserslautern.de". Domain of origin: "onlinetermine.kaiserslautern.de"
Nov. 06, 2023 3:56:17 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNUNG: Cookie rejected [rc_sid99f7e7c958a84cb084fd362b95200aaa="48f8b876f10b4be18c670eb8fb37998d7b3898197b76443bab04b27a7fae1e9f", version:0, domain:.callup.kaiserslautern.de, path:/m/Fuehrerscheinstelle, expiry:Mon Nov 06 16:16:13 CET 2023] Illegal 'domain' attribute ".callup.kaiserslautern.de". Domain of origin: "onlinetermine.kaiserslautern.de"
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler error
WARNUNG: CSS error: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [59:8762] Error in class selector. (Invalid token "and". Was expecting one of: "only", "inherit", <IDENT>.)
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler warning
WARNUNG: CSS warning: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [59:8762] Ignoring the whole rule.
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler error
WARNUNG: CSS error: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [59:17897] Error in class selector. (Invalid token "and". Was expecting one of: "only", "inherit", <IDENT>.)
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler warning
WARNUNG: CSS warning: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [59:17897] Ignoring the whole rule.
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler error
WARNUNG: CSS error: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [75:15911] Error in class selector. (Invalid token "and". Was expecting one of: "only", "inherit", <IDENT>.)
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler warning
WARNUNG: CSS warning: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [75:15911] Ignoring the whole rule.
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler error
WARNUNG: CSS error: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [75:62364] Error in class selector. (Invalid token "and". Was expecting one of: "only", "inherit", <IDENT>.)
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler warning
WARNUNG: CSS warning: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [75:62364] Ignoring the whole rule.
Nov. 06, 2023 3:56:18 PM org.htmlunit.WebConsole info
INFORMATION: [2023-11-06T14:56:18.683Z] WRN: module: clu.core is obselete use now rc.core, will removed in version 1.2.58
Nov. 06, 2023 3:56:19 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNUNG: Cookie rejected [rc_sid99f7e7c958a84cb084fd362b95200aaa="48f8b876f10b4be18c670eb8fb37998d7b3898197b76443bab04b27a7fae1e9f", version:0, domain:.callup.kaiserslautern.de, path:/m/Fuehrerscheinstelle, expiry:Mon Nov 06 16:16:15 CET 2023] Illegal 'domain' attribute ".callup.kaiserslautern.de". Domain of origin: "onlinetermine.kaiserslautern.de"
Nov. 06, 2023 3:56:19 PM org.htmlunit.WebConsole info
INFORMATION: [2023-11-06T14:56:19.143Z] WRN: module: clu.core is obselete use now rc.core, will removed in version 1.2.58
Nov. 06, 2023 3:56:24 PM org.htmlunit.html.DomElement click
WARNUNG: Calling click() ignored because the target element 'HtmlSpan[<span class="counterButton" onclick="changecap(1,2,'bd59a4e2-288d-4117-b6df-45c694d076fd')">]' is not displayed.
How do I make sure the span gets displayed, or rather clicked (also when not displayed)? I tried waiting for 5 seconds, but it changed nothing.
Maybe it has something to do with the fact that the span is in a div with scrollable overflow and the span is pretty much at the end.
Doing the layout and style calculation is sometimes difficult for a headless browser…
If your page is public available, please open an issue and i will try to find the reason and fix it if possible.
But there is a workaround – please have a look at this https://www.htmlunit.org/faq.html#clickNotWorking FAQ entry.
Hope that helps, if not please open an issue and i will have a look.
“Maybe it has something to do with the fact that the span is in a div with scrollable overflow and the span is pretty much at the end”: That’s probably it. You should tell htmlunit to scroll to the element first.