Bug,Feature or Design: With PHP cURL cookie needs value for proper caching

[serpent_driver]

I have developed a PHP based Multi-Threading crawler for cache warmup that uses cURL. This crawler also supports multiple cache varies. It works like it should and crawls up to 100,000 URLs within 1 hour, but cache varies drives my crazy. All cache varies work if I request URLs in browser with different cookies and they also work if I warmup the cache with cURL requests in shell script. For cache varies I use different and multiple cookies and also different user agents. Everything works fine, but not if I request URLs in PHP cURL requests with this crawler. For debugging I added cache status and cookies in access_log, but everything looked okay. After hours of testing I found out that cookie needs also cookie value in PHP cURL request.

CURLOPT_COOKIE => "cookie_1=;cookie_2=" // doesn't work

CURLOPT_COOKIE => "cookie_1=abc;cookie_2=def" // that works

So the question is, is this behaviour caused by PHP cURL or LSWS?

 

[winkybil]

Bit of a PHP n00b who’s been searching for an answer on how to create a proxy server to get around CORS issues. Ultimately, I’m trying to access an external API via a JavaScript fetch request and authenticate a login, set a cookie for future requests, etc.







Kodi nox

 

[AndreyPopov]

CURLOPT_COOKIE => "cookie_1=;cookie_2=" // doesn't work

CURLOPT_COOKIE => "cookie_1=abc;cookie_2=def" // that works

undefined variable is very very bad

if cookie is empty may be must be like this:

CURLOPT_COOKIE => " cookie_1='';cookie_2='' "

 

[serpent_driver]

This not an undefined varibale. Cookie name and its value is an expression. To set cookie with cURL without cookie value there is no need to set a value, but "=" must be set if only cookie name is set. This is RFC compliant.

Please don't talk about things you don't understand.

 

[AndreyPopov]

To set cookie with cURL without cookie value there is no need to set a value, but "=" must be set if only cookie name is set. This is RFC compliant.

but may be empty cookie must be sets by separate commands?

CURLOPT_COOKIE => "cookie_1="
CURLOPT_COOKIE => "cookie_2="

 

[serpent_driver]

NO! This is cURL and there may only be 1 CURLOPT_COOKIE. Otherwise only the last define is used and all others will be ignored.

Again, please don't talk about things you don't understand.

 

[AndreyPopov]

may be syntax must be right


CURLOPT_COOKIEThe contents of the "Cookie: " header to be used in the HTTP request. Note that multiple cookies are separated with a semicolon followed by a space (e.g., "fruit=apple; colour=red")

 

[serpent_driver]

It works with and without space after semicolon. The reason why it didn't work was a space before semicolon, so it is no LSWS issue.