XML downloads return HTTP 200 with “Request Rejected”

[carlos@agpsoftware.com]

Hello,

We use AntView 2 in a 32-bit Visual FoxPro Advanced application to browse Spain’s Public Sector Procurement Platform and download the XML documents associated with procurement notices.

We are investigating intermittent server rejections. We would like to confirm that we are using AntView’s browser session correctly and whether there is a more appropriate way to download these documents through the control.

ENVIRONMENT

- 32-bit Visual FoxPro Advanced. Reported version: Visual FoxPro 10.00.0000.2508.
- ActiveX control: AntViewAx2.AntView.
- Document interface: AntViewAx2.AntViewDocument.
- User-Agent observed in AntView:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.0.0 Safari/537.36 Edg/156.0.0.0

HOW THE APPLICATION WORKS

We first navigate to the procurement notice’s detail page using AntView. We associate the document object with the browser:

m.oDocument.CurrentBrowser = m.oBrowser

We then extract the “XML document” link from the page’s DOM. It points to an endpoint such as:

https://contrataciondelestado.es/FileSys...ntIdParam=...

To download the XML within the browser session, we execute JavaScript through RunAnonymousFunctionSync. The function starts an asynchronous fetch and immediately returns a “started” indicator. We then poll its status through subsequent calls to RunAnonymousFunctionSync.

OBSERVED PROBLEM

Initially, valid XML documents are downloaded. At some point, the document endpoint starts returning HTTP 200, but the response body is an HTML page titled “Request Rejected”, rather than XML.

The page contains:

The requested URL was rejected. Please consult with your administrator.
Your support ID is: <numeric identifier>

Passing this HTML to the XML parser produces an error equivalent to:

A name started with an invalid character.
Line 1, position 157.

We confirmed that the parsing error is caused by the numeric support identifier enclosed in < and > in the rejection page.

The procurement detail page remains accessible while the document download fails.

Our application makes an initial attempt followed by up to three retries, with one second between retries. If all four attempts fail, it logs the error and continues with the next notice. There is no additional deliberate delay between notices; they are processed sequentially.

TESTS PERFORMED

1. Direct download using MSXML2.ServerXMLHTTP.6.0, with a Chrome User-Agent header.
2. Download through fetch within the AntView session, using credentials: 'include'.
3. Closely timed comparisons of both methods against the same XML URL.
4. Tests from a new VFPA process and a fresh AntView session that first navigates to the detail page.
5. Verification that the rejected URLs are exactly the same URLs that previously returned valid XML.

Results on 5 November 2026, Madrid local time (UTC+1):

- At 12:31:10, both AntView and ServerXMLHTTP successfully downloaded the same document. The saved files had identical hashes.
- In a subsequent download run using AntView, 19 XML documents were retrieved successfully. Rejections began with notice 20, and subsequent documents also started failing.
- At 12:43:04–12:43:05, we repeated the comparison while the rejection was occurring. Both AntView and ServerXMLHTTP returned HTTP 200 with a “Request Rejected” HTML body. AntView received 245 characters and ServerXMLHTTP received 244.

We have also observed the rejection when manually opening the XML link in another browser. Therefore, we are not attributing the problem to AntView, and we have not established that bot detection is the specific cause. A server-side filtering rule may be involved, but we do not know which one.

QUESTIONS

1. Is this a correct use of RunAnonymousFunctionSync to start an asynchronous fetch and subsequently poll its result?

2. Does a fetch executed this way use the same session, cookies and network context as navigation within that AntView control?

3. Are there relevant differences between downloading through fetch and clicking the document link, particularly regarding cookies, redirects, Referer and navigation-related headers?

4. Would you recommend a particular AntView API or event for downloading the document and retrieving its bytes while preserving the browser context?

5. How can we capture the actual request and response headers, redirects and network events through AntView to compare a successful request with a rejected one?

6. What additional AntView/WebView2 version or configuration information would you need to investigate this case?

Our aim is to verify that we are using the control correctly and collect diagnostic information that we can also submit to the procurement platform’s support team, if necessary.

We can provide a minimal Visual FoxPro example, an affected public URL, response bodies and timestamped logs. We would exclude cookies and other session data from any shared files.

Thank you for your assistance.

Best regards,

Carlos Yohn
A.G.P. S.L.

moderator edit: moved all text out of the code section as it was particularly hard to read that way.

[wila]

Hi,

That's some pretty advanced use right there.

A few remarks.

To me it sounds very much like you are hitting some anti-scraping protection.
There's a few points that make me believe so.
- Hitting the exact same issue when using a completely different mechanism (MSXML2.ServerXMLHTTP.6.0)
- Have this happen on the 19th download every time
- Receiving a HTTP 200 status code (e.g. all OK) instead of a HTTP error.

What I would do in your situation is:
- use (Telerik) Fiddler for inspecting the data stream both requests and responses. It's a great tool for diagnosing this kind of issue and best of all, it's free.
- try and see if the same issue happens when you download via a browser manually. If it doesn't then try to increase time between the downloads to say 30 seconds.
- Log out of the page after for example 15 downloads then login again and continue to download the next 15 documents and so on.

Some answers to your questions..
- I think you are using AntView correctly
- The XML parsing error is expected if the page contains non XML data. You could probably test the content to see if it starts with "<" character after the white space.
- If you use javascript fetch then I don't think that the browser component itself can investigate any HTTP headers as the data is at the javascript layer (I might be wrong on this). Normally you can retrieve header data on document navigation events when the page is navigated. eg. OnNavigationStarting has the request object. But that's in the browser document.
There's also NavigateWithWebResourceRequest which can give you access to header data and both request and response objects via the OnWebResourceRequested event.
But if the issue is bot/scrape detection it won't give you anything extra over your current methods.

First try and see if a Fiddler snapshot can get you some more answers.

[carlos@agpsoftware.com]

Thank you for your suggestions and for confirming that our use of AntView looks correct.

We now have additional evidence:

- Manual XML downloads worked before starting the automated process.
- In our latest test, using a different public IP from the previous test, 66 XML downloads succeeded before the first rejection. The failure therefore does not consistently occur on download 19.
- Once the rejections started, manual downloads also failed in the browser, including when clicking the XML link directly on the notice page. Several different XML documents were affected.
- We then tested the same XML on a mobile phone: **it failed over the same Wi-Fi connection, but worked after switching to 5G**.

This strongly suggests a restriction associated with the public source IP, possibly triggered by request volume or frequency, rather than an AntView-specific issue. However, we cannot yet identify the exact rule.

In an earlier test, increasing the delays progressively to 60 seconds did not immediately restore access. A subsequent document was successfully downloaded on a retry approximately ten minutes after the first rejection.

We have reported these findings to the procurement platform’s support team and asked them to clarify their access restrictions and the supported approach for automated downloads.

We also plan to capture successful and rejected requests with Fiddler, as you suggested, to compare request and response headers, cookies, redirects and any diagnostic identifiers. Even if the restriction is IP-based, this should help us provide better evidence to their support team.

The rejected responses return HTTP 200 with an HTML “Request Rejected” page. Checking for an initial `<` character would therefore not distinguish them from XML; we need to validate the XML and its expected root element.

Thank you again for your help.