|
|
XVIII. Функции CURL, Client URL Library
В PHP включена поддержка libcurl - библиотеки функций, написанной
Daniel Stenberg, которая позволяет взаимодействовать с различными
серверами по различным протоколам. В настоящее время libcurl
поддерживает протоколы http, https, ftp, gopher, telnet, dict, file,
и ldap. libcurl также умеет работать с сертификатами HTTPS, посылать
запросы к HTTP серверам методами POST и PUT, закачивать файлы по
протоколам HTTP и FTP (последнее можно сделать также используя модуль FTP), использовать прокси-серверы,
cookies и аутентификацию пользователей.
Эти функции были добавлены в PHP 4.0.2.
Для использования функций CURL необходимо установить пакет CURL. Необходимо мспользовать версию CURL
7.0.2-beta или старше, с более младшими версиями PHP не работает. Для
PHP 4.2.3 необходимо использовать версию CURL 7.9.0 или старше.
Начиная PHP 4.3.0, необходима CURL 7.9.8 или старше. Вероятнее
всего, PHP 5.0.0 будет требовать CURL не младше 7.10.5.
Для использования CURL необходимо также собрать PHP с опцией --with-curl[=DIR], где DIR - имя директории,
содержащей поддиректории lib и include. Директория "include" должна
содержать поддиректорию "curl" с файлами easy.h и
curl.h. Директория "lib" должна
содержать файл libcurl.a. Начиная с PHP 4.3.0 с
помощью опции --with-curlwrappers
можно сконфигурировать PHP для использования CURL при работе с
удаленными файлами.
Замечание для пользователей Win32:
Для работы с этим модулем в Windows необходимо скопировать файлы
libeay32.dll и ssleay32.dll
из директории DLL Win32 дистрибутива PHP в директорию SYSTEM
(Нпример C:\WINNT\SYSTEM32
или C:\WINDOWS\SYSTEM)
Перечисленные ниже константы определены данным расширением и могут быть
доступны только в том случае, если PHP был собран с
поддержкой этого расширения или же в том случае, если
данное расширение подгружается во время выполнения.
После сборки PHP с поддержкой CURL можно использовать функции CURL
functions. Работа с CURL всегда начинается с вызова
curl_init(), затем устанавливаются необходимые
параметры с помощью curl_setopt(), и выполняется
требуемая операция вызовом curl_exec(), после чего
вызовом curl_close() сеанс работы завершается.
Приведенный ниже пример использует функции CURL для сохранения
стартовой страницы сайта example.com в файл:
Пример 1. Использования модуля CURL для сохранения стартовой страницы example.com |
<?php
$ch = curl_init("http://www.example.com/");
$fp = fopen("example_homepage.txt", "w");
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
fclose($fp);
?>
|
|
 add a note
User Contributed Notes
Функции CURL, Client URL Library
ryan at tankprofiler dot com
29-Oct-2007 01:26
If you have trouble on server 2003, IIS 6 ( perhaps other versions ) with getting the php_curl loading please see the following.
- run (as an administrator) php.exe -i > C:\phpinfo.txt and go open C:\phpinfo.txt, look in the file to see if CURL was loading, if it's there then keep reading.
- running <?PHP phpinfo(); ?> inside a text.php script on my IIS server would not show CURL loading
- A permissions problem on libeay32.dll and ssleay32.dll was causing the cli version of php allowing me access to these two dlls while IIS was not able to get them. I gave 'everyone' read and execute on these two dll's to try to fix the issue and it just now worked. You may wish to be more restrictive, perhaps IUSR rather than EVERYONE; but check your permissions none-the-less.
Hopefully this saves someone some time!
- Ryan Nanney
jfkallen at hotmail dot com
06-Oct-2007 11:54
There is a problem with uploading files via CURL. However the issues regarding the @ symbol as well as references to url_encoding are not the cause of the bug.
The @ symbol is used to send the contents of a file, rather than the string representation of the file path, to the receiving script. (See Below)
In other posts on this site references are made to CURLOPT_POSTFILEDS being populated with an array of post variables, however the documentation is clear, this variable is supposed to be a string,....therefore the following should work.
function Post($vars,$url){
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,1);
#curl_setopt($ch,CURLOPT_UPLOAD,1);
curl_setopt($ch,CURLOPT_POSTFIELDS,$vars);
curl_exec($ch);
curl_close($ch);
}
$path = "/tmp/test.txt";
$url = "http://www.somesite.com/receive_changes.php";
$vars = "action=submitted&order=receive&file=@$path";
Post($vars,$url);
Unfortunately this does not work.
Ojas Ojasvi
25-Sep-2007 10:52
<?
/*
* Author: Ojas Ojasvi
* Released: September 25, 2007
* Description: An example of the disguise_curl() function in order to grab contents from a website while remaining fully camouflaged by using a fake user agent and fake headers.
*/
$url = 'http://www.php.net';
// disguises the curl using fake headers and a fake user agent.
function disguise_curl($url)
{
$curl = curl_init();
// Setup headers - I used the same headers from Firefox version 2.0.0.6
// below was split up because php.net said the line was too long. :/
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // browsers keep this blank.
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)');
curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl, CURLOPT_REFERER, 'http://www.google.com');
curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($curl, CURLOPT_AUTOREFERER, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_TIMEOUT, 10);
$html = curl_exec($curl); // execute the curl command
curl_close($curl); // close the connection
return $html; // and finally, return $html
}
// uses the function and displays the text off the website
$text = disguise_curl($url);
echo $text;
?>
Ojas Ojasvi
vuk at REMOVE_THIS_PART qstatistic.com
20-Sep-2007 09:09
a few notes regarding the paymentech solution posted above by "ksavage". to work properly, two changes are required (at least they were for my system/account).
first, the content-type (in header) should read:
"Content-type: application/PTI40\r\n";
.. and NOT (PTI26)
second, the following needs to be added to the script (where the curl options are stated):
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0)
...otherwise, paymentech will ask for SSL certificate.
bensan dot benny at vrs-soft dot com
04-Sep-2007 07:35
I would like to specify a content-type for a file I am uploading with cURL and yet I find I do not have any way of doing this. I know that there is a mechanism through the command-line cURL to specify content-types of files.
I want to set userInfo.xml to have a Content type of "text/xml". Here is my code:
<?php
$postData['userInfo'] = "@userInfo.xml";
$url = "http://curl.dev";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
$response = curl_exec($ch);
print_r($response);
if(curl_errno($ch))
{
print curl_error($ch);
}
curl_close($ch);
?>
charles at tastik dot net
01-Sep-2007 02:27
FYI,
Anyone trying to connect to .NET with CURL to send a simple XML post, pay attention to the following. This will save you hours! There is a previous note that I saw either on this page, or somewhere else on this site that explains the correct way to specify the header option is to create an array, then reference the array from the CURLOPT.
ie. Do something like this:
// Req. HTTP Header Values
$header[] = "Content-type: text/xml";
// Target URL
$sendTo = "http://www.blah.com";
// Post Data
$post = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<root>\n....etc, etc,";
// Create CURL Connection
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, 'XtraDoh xAgent');
curl_setopt($ch, CURLOPT_URL, $sendTo);
curl_setopt($ch, CURLOPT_TIMEOUT, 900);
curl_setopt($ch, CURLOPT_CONNECTIONTIMEOUT, 30);
curl_setopt($ch, CURLOPT_FAILONERROR, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
Notice the HTTPHEADER, $header above. I have not been able to get .NET to properly read the HTTP header as specified (in this case as text/xml) when using the following:
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type'=>'text/xml'));
Although this may work when working with other PHP, IIS, or even PHP, Apache, it does not (at least in my experience) work with .NET, IIS.
mtsoft at mt-soft dot com dot ar
09-Jul-2007 02:30
This function takes a URL and returns a plain-text version of the page. It uses cURL to retrieve the page and a combination of regular expressions to strip all unwanted whitespace. This function will even strip the text from STYLE and SCRIPT tags, which are ignored by PHP functions such as strip_tags (they strip only the tags, leaving the text in the middle intact).
Regular expressions were split in 2 stages, to avoid deleting single carriage returns (also matched by \s) but still delete all blank lines and multiple linefeeds or spaces, trimming operations took place in 2 stages.
function webpage2txt($url)
{
$user_agent = “Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)”;
$ch = curl_init(); // initialize curl handle
curl_setopt($ch, CURLOPT_URL, $url); // set url to post to
curl_setopt($ch, CURLOPT_FAILONERROR, 1); // Fail on errors
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // allow redirects
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); // return into a variable
curl_setopt($ch, CURLOPT_PORT, 80); //Set the port number
curl_setopt($ch, CURLOPT_TIMEOUT, 15); // times out after 15s
curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
$document = curl_exec($ch);
$search = array(’@<script[^>]*?>.*?</script>@si’, // Strip out javascript
‘@<style[^>]*?>.*?</style>@siU’, // Strip style tags properly
‘@<[\/\!]*?[^<>]*?>@si’, // Strip out HTML tags
‘@<![\s\S]*?–[ \t\n\r]*>@’, // Strip multi-line comments including CDATA
‘/\s{2,}/’,
);
$text = preg_replace($search, “\n”, html_entity_decode($document));
$pat[0] = “/^\s+/”;
$pat[2] = “/\s+\$/”;
$rep[0] = “”;
$rep[2] = ” “;
$text = preg_replace($pat, $rep, trim($text));
return $text;
}
Potential uses of this function are extracting keywords from a webpage, counting words and things like that. If you find it useful, drop us a comment and let us know where you used it.
Mr.KTO
07-Jul-2007 06:59
Don't foget to curl_close($ch); Even if curl_errno($ch) != 0
Because if you don't - on Windows this will produce windows-error-report (Program terminated unexpectedly)
ashw1 - at - no spam - post - dot - cz
03-Jun-2007 04:51
In case you wonder how come, that cookies don't work under Windows, I've googled for some answers, and here is the result: Under WIN you need to input absolute path of the cookie file.
This piece of code solves it:
<?php
if ($cookies != '')
{
if (substr(PHP_OS, 0, 3) == 'WIN')
{$cookies = str_replace('\\','/', getcwd().'/'.$cookies);}
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookies);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookies);
}
?>
pyroevi at yahoo dot com
30-May-2007 04:58
Earlier I posted about using php to interface with an authorize.net gateway. I accidentally made a mistake in my code.
Instead of:
curl_setopt($connection, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($AuthNetConn, CURLOPT_CAINFO, "path:/ca-bundle.crt");
It should be
curl_setopt($connection, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($connection, CURLOPT_CAINFO, "path:/ca-bundle.crt");
Whatever you call that first connection variable, it needs to be the same in both statements. Sorry about that.
Mr.KTO
17-May-2007 07:43
There is a bug in file uploading using multipart/form-data. (WinXP, PHP 4.4.4, libcurl/7.14.0)
curl_setopt($ch, CURLOPT_POSTFIELDS, array("userfile" => "@filename.ext");
produces error "failed creating formpost data"
Solution - use absolute path for uploaded file. As:
curl_setopt($ch, CURLOPT_POSTFIELDS, array("userfile" => '@'.realpath('filename.ext'));
p.s: Also slash before '@' don't escape it, so post values with first '@' by multipart mothod not supported.
rk6314 at gmail dot com
16-May-2007 12:25
I also think that a leading ampersand is a special escape for the cURL library to tell it that this post field is actually a path to a file that should be sent to the web server.
Not sure if that works yet, but I sure hope so. I don't relish the thought of rolling my own multi-part submitter by hand for something I'm working on.
trucex at gmail dot com
13-May-2007 08:51
Many people still use the fopen wrappers, file_get_contents(), or even include() to open files across the internet. The issue with this is that they were not designed to do so and they are inefficient in doing so. Curl is a great alternative to these and isn't much harder to use. It even beats out the use of sockets in fairly default configurations.
I have created a script that tests all 4 methods on 20 different websites 10 times each. The results are clear, here is one set of the results:
Curl time: 43.02 seconds
FGC time: 86.48 seconds
Fopen time: 86.34 seconds
Socket time: 44.91 seconds
As you can see, there is little performance difference between file_get_contents and fopen/fread. Likewise, there is little difference between curl and sockets.
You may use the test to see the results in realtime if you'd like: http://trucex.com/curltest.php
To Frank DAVIS
09-May-2007 03:54
Post variables should be properly urlencode()'d, so, for your example,
$data = array('text' => urlencode(stripslashes("@pouetpouet")));
Frank DENIS
28-Apr-2007 02:41
There's a serious bug with CURLOPT_POSTFIELDS and multipart data :
<?php
$data = array('text' => "@pouetpouet");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://127.0.0.1:9999');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_exec($ch);
?>
You can grab the output with something like "nc -l -p 9999" (Linux) or "nl -l 9999" (OpenBSD).
If the text begins with a @ (@pouetpouet), nothing is sent. If the text doesn't begin with @, the request is properly sent.
Another issue is that there's a slight delay between the header and the first block of multipart data.
killermonk at REMOVE dot killermonk dot com
30-Mar-2007 02:57
I would like to point out that in my post below "button_name" is used as the name="" attribute of the submit image.
So if you had the following code:
<input type="image" name="cmd_login" onclick="if (typeof(Page_ClientValidate) == 'function') Page_ClientValidate(); " language="javascript" id="cmd_login" src="/images/cmd_login.gif" alt="Login" border="0" />
then you would use "cmd_login.x" and "cmd_login.y"
killermonk at REMOVE dot killermonk dot com
06-Mar-2007 11:23
For anyone trying to use cURL to submit to an ASP/ASPX page that uses an image as the submit button.
Make sure that you have 'button_name.x' and 'button_name.y' in the post fields. PHP names these fields 'button_name_x' and 'button_name_y', while ASP uses a dot.
Also, as noted above, be sure to include the '__VIEWSTATE' input field in your post request.
gordon_e_rouse at yahoo dot com dot au
25-Jan-2007 02:24
Using curl to take snapshots of the current page for emailing the HTML is a clever little idea. (ie: Email this page to a friend)
//to be explained below!
session_write_close();
$pageurl = "http://www.site.com/content.php?PHPSESSID=123XYZ
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_URL, $pageurl );
$html = curl_exec ( $ch );
curl_close($ch);
//then you need to fix pathing to absolute
$search = "/(src|href|background)=\"[^:,^>,^\"]*\"/i";
preg_match_all ( $search, $html, $a_matches );
//you can figure out the rest ! but thought the reg expression is useful as well
But here is the catch, you may want to make sure curl connects to the server under the same session as the browser. So naturally you pass the session cookie through the curl system either by the cookie jar system, or through the query string in the path.
This is where you will get stuck. PHP will need write access to the same session file simultaneously!! causing serious hanging issues!
This is why you should close off your session before you make curl take a page snapshot!
michael (at) networkstuff co nz
20-Jan-2007 01:05
If PHP configure fails with cURL errors, try ommiting the --with-curl=path and just make this --with-curl.
Of course it will also be optimal to make sure that the cURL library directory is listed in /etc/ld.so.conf and then run 'ldconfig'.
By default this is /usr/local/lib.
null at nahdah dot com
27-Nov-2006 01:42
I was having problems with Authorize.net and the SSL cert matching even after adding:
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
What I found after a lot of stumbling was I needed to set VERIFYHOST to FALSE. So if you are still have a problem with Authorize.NET SSL and cURL add this:
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
pyroevi at yahoo dot com
22-Nov-2006 10:33
I was working on using php to interface with an authorize.net gateway, and I ran into a problem with certificates using curl to talk the https:// url.
curl_error() told me "SSL certificate problem, verify that the CA cert is OK."
I googled it and found the same "solution" over and over again: bypass verification by adding this line after curl_init():
curl_setopt($connection, CURLOPT_SSL_VERIFYPEER, false);
This worked great, but I was required to verify, so here's what I did. Add the following lines:
curl_setopt($connection, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($AuthNetConn, CURLOPT_CAINFO, "path:/ca-bundle.crt");
with "path:/ca-bundle.crt" being the path to that certificate file. You can get this file by downloading the curl package (http://curl.haxx.se/download.html) and looking for it in the lib folder.
Feel free to email me.
jon at jonroig--- dot com
25-Aug-2006 06:53
For anyone who is having trouble getting some of the advanced functionality to work in whatever version of PHP you have, I wrote a little wrapper for the command line version of curl. The function below is for Windows (hence, the curl.exe), but it works the same way under Linux or whatever.
The curl man page enumerates a ton of options you can use...
function curlPageGrabber($destinationURL, $refererURL, $postData, $autoForward = false, $autoGrabCookies = true, $curlDebug = false)
{
$curlString = "c:\curl\curl.exe -i -v -m 30 -L -b c:\curl\cookiejar.txt -c c:\curl\cookiejar.txt ";
$curlString .= " -A \"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322;)\" ";
$curlString .= " -H \"Accept-Language: en-us\" ";
$curlString .= " -H \"Accept-Encoding: gzip, deflate\" ";
$curlString .= " -H \"Host: affiliate-program.amazon.com\" ";
$curlString .= " -H \"Accept: */*\" ";
$curlString .= " --compressed ";
//$curlString .= " --trace-ascii ";
if ($refererURL != '')
{
$curlString .= "-e \"".$refererURL."\" ";
}
if ($postData != '')
{
$curlString.= " -d \"$postData\" ";
}
$curlString .= $destinationURL;
if ($curlDebug == true)
{
echo "<p>curlString: $curlString</p>";
}
$temp = exec($curlString, $retval);
$htmlData = '';
foreach ($retval as $arrayLine)
{
$htmlData .= $arrayLine."\r\n";
}
if ($curlDebug == true)
{
echo "<textarea cols=80 rows=20>$htmlData</textarea>";
}
return $htmlData;
} // end function
galvao at galvao.eti.br
06-Jul-2006 08:27
Note that on Win32 this documentation can get a little confusing.
In order to get this to work you need to:
1) Be sure that the folder where libeay32.dll and ssleay32.dll - tipically C:\\PHP - is present on the PATH variable.
2) Uncomment - remove the semi-colon - the line that says "extension=php_curl.dll" from php.ini
3) Restart the webserver (you should already know this one, but...)
It took me some time to realize this, since this page doesn't mention the need to uncomment that php.ini's line.
david at neocodesoftware dot com
30-May-2006 05:42
This may be obvious to everybody *except* me, but if you want to use curl to connect via ftp rather than http, then you just need to use "ftp://" in the url specification (I was looking for an use_ftp flag or something).
Use the CURLOPT_USERPWD to login to the ftp site.
Sean Huber
05-May-2006 04:01
<?php
class CURL {
var $callback = false;
function setCallback($func_name) {
$this->callback = $func_name;
}
function doRequest($method, $url, $vars) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
if ($method == 'POST') {
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $vars);
}
$data = curl_exec($ch);
curl_close($ch);
if ($data) {
if ($this->callback)
{
$callback = $this->callback;
$this->callback = false;
return call_user_func($callback, $data);
} else {
return $data;
}
} else {
return curl_error($ch);
}
}
function get($url) {
return $this->doRequest('GET', $url, 'NULL');
}
function post($url, $vars) {
return $this->doRequest('POST', $url, $vars);
}
}
?>
tom at luigjes dot org
29-Mar-2006 02:48
When you try to login to a page using cURL, you could experience problems when you're not providing the right URL.
For instance, if you use the front page to login (http://awebsite.com/) where to form is located, you will receive a http 405 error, instead view the source and see what the form's action is, it might just be: http://awebsite.com/processlogin.php
Took me hours to find this out, hope it can save someone else's time.
richardkmiller AT gmail
02-Mar-2006 04:21
Beware of any extra spaces in the URL. A trailing space in the URL caused my script to fail with the message "empty reply from server".
heyrocker at yahoo dot com
23-Feb-2006 01:57
The examples below for HTTP file upload work great, but I wanted to be able to post multiple files through HTTP upload using HTML arrays as specified in example 38.3 at
http://us3.php.net/features.file-upload
In this case, you need to set the arrays AND keys in the $post_data, it will not work with just the array names. The following example shows how this works:
<?php
$post_data = array();
$post_data['pictures[0]'] = "@cat.jpg";
$post_data['pictures[1]'] = "@dog.jpg";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://my.domain.com/my_url.php" );
curl_setopt($ch, CURLOPT_POST, 1 );
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$postResult = curl_exec($ch);
if (curl_errno($ch)) {
print curl_error($ch);
}
curl_close($ch);
print "$postResult";
?>
stevehartken at hotmail dot com
21-Feb-2006 10:40
I had the following experience when harvesting urls with a get variable from a page using cUrl. HTML pages will output ampersands as & when the page is read by the curl function.
If you code a script to find all hyperlinks, it will use & instead of &, especially using a regular expression search.
It is hard to detect because when you output the url to the browser it renders the html. To fix, add a line to replace the & with &.
function processURL($url){
$url=str_replace('&','&',$url);
$ch=curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
$xml = curl_exec ($ch);
curl_close ($ch);
echo $xml;
}
}
petelu $ post dot sk
17-Feb-2006 04:19
load https:// or http://ex.com/exam.php
with POST data (name=alex&year=18) and apply COOKIEs
$sessions = curl_init();
curl_setopt($sessions,CURLOPT_URL,'http://ex.com/exam.php');
curl_setopt($sessions, CURLOPT_POST, 1);
curl_setopt($sessions,CURLOPT_POSTFIELDS,'name=alex&year=18');
curl_setopt($sessions,CURLOPT_COOKIEJAR,
dirname(__FILE__).'/cookie.txt');
curl_setopt($sessions,CURLOPT_FOLLOWLOCATION,0);
curl_setopt($sessions, CURLOPT_HEADER , 1);
curl_setopt($sessions, CURLOPT_RETURNTRANSFER,1);
$my_load_page = curl_exec($this->sessions);
amandato (at) gmail (period) com
06-Feb-2006 01:53
To shed some light on some of the numbers trucex at gmail has in his tests, some of the web pages (Yahoo and GoDaddy) downloaded at exceptionally faster speeds than with file_get_contents(). This is simply because CURL accepts (by default) compression protocols such as deflate and gzip.
Most likely GoDaddy.com may be using compression for specific HTTP clients including CURL. I know for a fact that Yahoo.com and Google.com use gzip compression. You can test servers for gzip compression here: http://www.whatsmyip.org/mod_gzip_test/
For the sake of performance, I would recommend the use of CURL. if performance was not an issue and I wanted to keep my code simple and portable (for PHP servers without the cURL extension) I would use file_get_contents().
luca dot manzo at bbsitalia dot com
02-Feb-2006 03:55
If you're getting trouble with cookie handling in curl:
- curl manages tranparently cookies in a single curl session
- the option
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookieFileName");
makes curl to store the cookies in a file at the and of the curl session
- the option
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/cookieFileName");
makes curl to use the given file as source for the cookies to send to the server.
so to handle correctly cookies between different curl session, the you have to do something like this:
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE_PATH);
curl_setopt ($ch, CURLOPT_COOKIEFILE, COOKIE_FILE_PATH);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec ($ch);
curl_close($ch);
return $result;
in particular this is NECESSARY if you are using PEAR_SOAP libraries to build a webservice client over https and the remote server need to establish a session cookie. in fact each soap message is sent using a different curl session!!
I hope this can help someone
Luca
|
ctype_xdigit