# [SOLVED] PHP HTTP Functions / fsockopen

## midnite

i am not sure if i should post this thread here. But i am sure that some of the members here runs a PHP server would come across my problem and probably have some solutions  :Wink: 

i run Apache, (on Gentoo, of course).

Very simple, i want to use PHP HTTP functions (http://hk2.php.net/manual/en/ref.http.php).

Why? i want to use PHP to login and fill in a form. Probably http_post_data() is the best choice. Any other solutions?

Yet i have the following error when i access the PHP page: *Quote:*   

> Fatal error: Call to undefined function http_get() in /var/www/localhost/htdocs/testing.php on line 9

 

i have studied the PHP HTTP function installation & configuration and i guess, i still have not install that plugin. So i checked the USE flags:

```
# emerge -pv php

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild   R   ] dev-lang/php-5.2.6_rc4  USE="apache2 berkdb bzip2 cgi cjk crypt ctype curl curlwrappers exif force-cgi-redirect ftp imap ipv6 ldap ldap-sasl mysql mysqli nls pic readline session sharedext sharedmem simplexml snmp sockets spell spl ssl tidy tokenizer truetype unicode xml xmlreader xmlwriter zip zlib -adabas -bcmath -birdstep -calendar -cdb -cli -concurrentmodphp -db2 -dbase -dbmaker -debug -discard-path -doc -empress -empress-bcs -esoob -fastbuild -fdftk -filter -firebird -flatfile -frontbase -gd -gd-external -gdbm -gmp -hash -iconv -inifile -interbase -iodbc (-java-external) -json -kerberos -libedit -mcve -mhash -msql -mssql -ncurses -oci8 (-oci8-instant-client) -odbc -pcntl -pcre -pdo -posix -postgres -qdbm -recode -reflection -sapdb -soap -solid -sqlite -suhosin -sybase -sybase-ct -sysvipc -threads -wddx -xmlrpc -xpm -xsl -yaz -zip-external" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB
```

i really have no clue what is the next step i should do   :Sad: 

----------

## Desintegr

You need dev-php5/pecl-http.

----------

## midnite

 *Desintegr wrote:*   

> You need dev-php5/pecl-http.

 

thanks!!  :Very Happy: 

sorted   :Razz: 

----------

## midnite

one more question, can they manage HTTPS connections?

----------

## Erulabs

 *Quote:*   

> one more question, can they manage HTTPS connections?

 

As far as I know dev-php5/pecl-http cannot.

You can do it the manual way and hack together some sockets.

With sockets you'd do something like 

```
$fp = fsockopen("ssl://127.0.0.1", 443, $errno, $errstr);
```

See http://us2.php.net/function.fsockopen.

Hope this helps! Sockets rule by the way. Its more than possible to write an entire webserver app in pure PHP using sockets in under 20 lines of code. (Who said PHP wasn't concise?)

----------

## midnite

 *Erulabs wrote:*   

>  *Quote:*   one more question, can they manage HTTPS connections? 
> 
> As far as I know dev-php5/pecl-http cannot.
> 
> You can do it the manual way and hack together some sockets.
> ...

 

Thanks Erulabs.

If it cannot handle HTTPS, it will throw an error? or else?

It is strange that (with pecl-http functions by GET or POST) i seem getting the correct page back. But when i check in Firefox Tamper Data, i should have a SESSID cookie. But i cannot find it in the string returned by pecl-http function. =(

If pecl-http function works, it will be very fine. i will try your socket approach later but i am not familiar with sockets.

Thanks =)

----------

## midnite

Hi Erulabs,

Thanks very much!!!! Really a million THANKS!!!!

i am now working on sockets. i am now halfway success. i have succeeded logged in and accessing forms by forms in my account!! And the most thrilling point is, i can see the future. i can see i am very near to success. i believe there will be no more difficulties once i manage to submit one form, i can submit as many forms as i want!!

Really thanks a lot!!

cheers =D

----------

## midnite

i have succeed my whole project!! COOL!!

but one thing i would like to say, fsockopen then fgets is much slower than file_get_contents or http_get. (Or i don't know how to use?)

----------

## jure1873

I've triedn doing this with php but it's easier with wget - you can tell it to save cookies in a file, post form fields, etc... if you're trying something easy like just submitting a form you can also try wget

----------

## Erulabs

midnite:

Yeah, sockets are very cool. I would guess your problem with sockets taking longer has to do with the remote server. Try sending "Connection: close" in your headers.

Heres a demo you might find interesting:

```
  $ip = "127.0.0.1";

  $port = "443";

  $post = "this=that&thisotherthing=thattoo\n";

  $head .= "POST /path/to/file/to/post/to HTTP/1.1\n";

  $head .= "Host: who.were.talking.to\n";

  $head .= "Connection: close\n";

  $head .= "User-Agent: PHPscript\n";

  $head .= "Cookie: acceptCookies=true;\n";

  $head .= "Content-Type: application/x-www-form-urlencoded\n";

  $head .= "Content-Length: ".strlen($post)."\n\n";

  $head .= $post;

  $open = fsockopen("ssl://".$ip,$port);

  fputs($open, $head);

  stream_set_blocking($open, TRUE);

  stream_set_timeout($open,7);

  $info = stream_get_meta_data($open);

  $read = fread($open,209);

  eregi("Set-Cookie:.*", $read, $setcookie);

  $session = substr($setcookie[0], 23, 32);

 

  while ((!feof($open)) AND (!$info['timed_out'])) {

      $stats .= fgets($open, 1496);

      $info = stream_get_meta_data($open);

      flush();

  }

```

This allows you to post information to an ssl (https) protected server, and it does it fast =D

Note here that $session is going to be your JSESSION id. You'll probably need to modify this if you're talking to anything but the lame JSP server I coded this for.

Also, $read is the HTTP headers the server responds with (I only needed the first 209 bytes).

Another bit of advice, check out https://addons.mozilla.org/en-US/firefox/addon/3829 for all your http header capturing needs

----------

## midnite

Thanks Erulabs!

i would like to ask what is the different between "Connection: close" and "Connection: keep-alive"? i used the Firefox Tamper Data plugin and found that it is sending "Connection: keep-alive". So i put "Connection: keep-alive" in my PHP scripts to simulate the POST request. Is it one of the point that makes my script slow?

And also, why you know you need only 209 bytes? Is it a standard length of the response header? Or it just apply to your server?

i see you use "flush()". Will it be faster? Or slower as it perform one more action?

Here's my script:

```
define ('debug', false);

define ('sslSRC_host', 'www.domain.com');

define ('sslSRC_action', '/login_page');

define ('sslSRC_term', '/index_page');

function addout (&$o) {

  $o .= 'Host: '.sslSRC_host."\r\n";

  $o .= 'User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14'."\r\n";

  $o .= 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'."\r\n";

  $o .= 'Accept-Language: en-gb'."\r\n";

  $o .= 'Accept-Encoding: gzip,deflate'."\r\n";

  $o .= 'Accept-Charset: UTF-8,*'."\r\n";

  $o .= 'Keep-Alive: 300'."\r\n";

  $o .= 'Connection: keep-alive'."\r\n";

}

function getSESSID (&$p) {

  while (!feof($p)) {

    $ws = fgets($p, 128);

    if ($pos = stripos ($ws, 'SESSID=')) return substr ($ws, $pos, stripos ($ws, "\r\n")-$pos);

  }

  return false;

}

$POST_DATA = 'data1=one&data2=two';

// submit login (socket)

echo '<h1>submit login (socket)</h1>';

$out = 'POST '.sslSRC_action." HTTP/1.1\r\n";

addout ($out);

$out .= 'Referer: https://www.domain.com/login_page'."\r\n";

$out .= 'Content-Type: application/x-www-form-urlencoded'."\r\n";

$out .= 'Content-Length: '.strlen ($POST_DATA)."\r\n\r\n";

$out .= $POST_DATA."\r\n";

if (debug) echo $out.'<hr />';

if (!$fp = fsockopen ('ssl://'.sslSRC_host, 443)) exit ('fsockopen failed');

fwrite($fp, $out);

if (!$SESSID = getSESSID ($fp)) exit ('getSESSID failed');

if (debug) echo $SESSID.'<hr />';

fclose ($fp);

// submit term (socket)

echo '<h1>submit term (socket)</h1>';

$out = 'POST '.sslSRC_term." HTTP/1.1\r\n";

addout ($out);

$out .= 'Referer: https://www.domain.com/index_page'."\r\n";

$out .= 'Cookie: '.$SESSID."\r\n";

$out .= 'Content-Type: application/x-www-form-urlencoded'."\r\n";

$out .= 'Content-Length: 14'."\r\n\r\n";

$out .= 'term_in=200806'."\r\n";

if (debug) echo $out.'<hr />';

if (!$fp = fsockopen ('ssl://'.sslSRC_host, 443)) exit ('fsockopen failed');

fwrite($fp, $out);

if (!$SESSID = getSESSID ($fp)) exit ('getSESSID failed');

if (debug) { echo $SESSID.'<hr />'; printTable ($fp, 'SUMMARY="Current Schedule"'); echo '<hr />'; }

fclose ($fp);

...

...
```

i have several forms to submit. As you can see above, i have to close the first connection and reopen another connection again. Is there a way not to close and reopen again?

Thanks =)

----------

## Erulabs

midnite:

You don't need to fclose() and then fsockopen() again.. you can just keep one connection going the entire time.

Also, the difference between "Connection: close" and "Connection: keep-alive" has to do with how the browser and web-server communicate. Things like active DOM, AJAX, and other technologies need a "keep-alive" connection to perform. Browsers like Mozilla FireFox wait until they receive an end of data command from the server, rather than just asking for a truck load of data. In the case of a PHP socket, a "keep-alive" request is going to be 99.9% unnecessary. There are a few complex actions you can take in terms of CGI communications with the server - I would doubt you'll need them.

As for the 209 bytes, no, thats no standard. I used that because thats everything up to the html data in the script I was using. You'll probably find you wont need more than 300 bytes of HTTP headers.

flush() "Flushes the output buffers of PHP and whatever backend PHP is using (CGI, a web server, etc). This effectively tries to push all the output so far to the user's browser.".

This is used in congress with "connection: close" to get the data from the webserver as quickly as possible.

Also, the use of stream_get_meta_data is used to determine if the connection has timed out while we are fetching data from it (as to prevent our script from needing to timeout on its own)

```
while ((!feof($open)) AND (!$info['timed_out'])) {

      $stats .= fgets($open, 1496);

      $info = stream_get_meta_data($open);

      flush();

  }
```

One other note, "\r" is evil. I can almost promise you'll have an easier time skipping them all together. Try to follow a strict standard for HTTP requests:

GET/POST /what HTTP/version {LINEBREAK '\n'}

HOST: Who.were.talking.to {LINEBREAK '\n'}

User-Agent: MyFakeBrowser {LINEBREAK '\n'}

Content-Length: LENGTH OF POST DATA {LINEBREAK '\n'}

{LINEBREAK '\n'}

POST DATA {LINEBREAK '\n'}

----------

## midnite

Thanks Erulabs for your detailed explanation =)

i also believe that i don't need to fclose() and fsockopen() again. But i got something interesting. i randomly found a simple webpage for demo: www.something.com

Here's my simple code:

```
<?php

function sendheader () {

$out = 'GET / HTTP/1.1'."\r\n";

$out .= 'Host: www.something.com'."\r\n";

$out .= 'User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14'."\r\n";

$out .= 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'."\r\n";

$out .= 'Accept-Language: en-gb'."\r\n";

$out .= 'Accept-Encoding: gzip,deflate'."\r\n";

$out .= 'Accept-Charset: UTF-8,*'."\r\n";

$out .= 'Keep-Alive: 300'."\r\n";

$out .= 'Connection: keep-alive'."\r\n\r\n";

return $out;

}

// first time

echo '<h1>first time</h1>';

if (!$fp = fsockopen ('www.something.com', 80)) exit ('fsockopen failed');

fwrite($fp, sendheader ());

while (!feof($fp))

  echo fgets ($fp);

// again

echo '<h1>again</h1>';

// without fsockopen here

fwrite($fp, sendheader ());

while (!feof($fp))

  echo fgets ($fp);

fclose ($fp);

?>
```

and this what i get: *Quote:*   

> <h1>first time</h1>HTTP/1.1 200 OK
> 
> Date: Sun, 08 Jun 2008 18:06:48 GMT
> 
> Server: Apache/1.3.34 (Unix)
> ...

 Note that the second response is lost.

But if i alter my code a bit:

```
<?php

function sendheader () {

$out = 'GET / HTTP/1.1'."\r\n";

$out .= 'Host: www.something.com'."\r\n";

$out .= 'User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14'."\r\n";

$out .= 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'."\r\n";

$out .= 'Accept-Language: en-gb'."\r\n";

$out .= 'Accept-Encoding: gzip,deflate'."\r\n";

$out .= 'Accept-Charset: UTF-8,*'."\r\n";

$out .= 'Keep-Alive: 300'."\r\n";

$out .= 'Connection: keep-alive'."\r\n\r\n";

return $out;

}

// first time

echo '<h1>first time</h1>';

if (!$fp = fsockopen ('www.something.com', 80)) exit ('fsockopen failed');

fwrite($fp, sendheader ());

for ($i=0; $i<5; $i++) // < ---------- changed here ONLY

  echo fgets ($fp);

// again

echo '<h1>again</h1>';

// without fsockopen here

fwrite($fp, sendheader ());

while (!feof($fp))

  echo fgets ($fp);

fclose ($fp);

?>
```

Then the second response will be append to the end of the first response: *Quote:*   

> <h1>first time</h1>HTTP/1.1 200 OK
> 
> Date: Sun, 08 Jun 2008 18:29:32 GMT
> 
> Server: Apache/1.3.34 (Unix)
> ...

 Note that there are TWO responses.

So, i have a conclusion that if i did not exhauste the file pointer $fp to end-of-file, it will append. Otherwise, not.

But why?

And i found a different behavior between "Connection: close" and "Connection: keep-alive":

```
<?php

function sendheader () {

$out = 'GET / HTTP/1.1'."\r\n";

$out .= 'Host: www.something.com'."\r\n";

$out .= 'User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14'."\r\n";

$out .= 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'."\r\n";

$out .= 'Accept-Language: en-gb'."\r\n";

$out .= 'Accept-Encoding: gzip,deflate'."\r\n";

$out .= 'Accept-Charset: UTF-8,*'."\r\n";

$out .= 'Keep-Alive: 300'."\r\n";

$out .= 'Connection: close'."\r\n\r\n"; // < -------------- changed here ONLY

return $out;

}

// first time

echo '<h1>first time</h1>';

if (!$fp = fsockopen ('www.something.com', 80)) exit ('fsockopen failed');

fwrite($fp, sendheader ());

for ($i=0; $i<5; $i++) // < ---------- changed here ONLY

  echo fgets ($fp);

// again

echo '<h1>again</h1>';

// without fsockopen here

fwrite($fp, sendheader ());

while (!feof($fp))

  echo fgets ($fp);

fclose ($fp);

?>
```

Then the second response will be lost again: *Quote:*   

> <h1>first time</h1>HTTP/1.1 200 OK
> 
> Date: Sun, 08 Jun 2008 18:38:19 GMT
> 
> Server: Apache/1.3.34 (Unix)
> ...

 There is really a lot to learn about HTTP connections :-)

And one more question: what does the 4e and 0 mean? i guess 4e should be a hex. It means the content has 78 bytes in total?

Is it true that if i use connection: close, i have to check $info['timed_out']? So i will risk connection timeout and lead to my process failure. If yes, i think i will stick to connection: keep-alive. Slower, but i will have better guarantee to success =)

Why "\r" is evil? i use \r\n because i found the response header uses \r\n too. But anyway, i believe you and try to follow your method. Yet, does HOST: Who.were.talking.to {LINEBREAK '\n'} means echo 'HOST: Who.were.talking.to {LINEBREAK \'\n\'}'; or echo 'HOST: Who.were.talking.to '.{LINEBREAK '\n'};?

Lots of questions, sorry =P

Thanks a lot, again =)

----------

