抓取页面常用方法是Curl，本文介绍另一个很好用的类。官网：
本文来自Snoopy官方文档，一起来学习下：
Snoopy - the PHP net client v2.0.0
SYNOPSIS(简单示例):
include "Snoopy.class.php";
$snoopy = new S
$snoopy-&fetchtext("http://www.php.net/");
print $snoopy-&
$snoopy-&fetchlinks("/");
print $snoopy-&
$submit_url = "http://lnk.ispi.net/texis/scripts/msearch/netsearch.html";
$submit_vars["q"] = "amiga";
$submit_vars["submit"] = "Search!";
$submit_vars["searchhost"] = "Altavista";
$snoopy-&submit($submit_url,$submit_vars);
print $snoopy-&
$snoopy-&maxframes=5;
$snoopy-&fetch("http://www.ispi.net/");
echo "&PRE&\n";
echo htmlentities($snoopy-&results[0]);
echo htmlentities($snoopy-&results[1]);
echo htmlentities($snoopy-&results[2]);
echo "&/PRE&\n";
$snoopy-&fetchform("");
print $snoopy-&
DESCRIPTION:
What is Snoopy?
Snoopy is a PHP class that simulates a web browser. It automates the
task of retrieving web page content and posting forms, for example.
史努比是一个模拟web浏览器的PHP类。它能实现自动搜索网页内容和提交表单。
Some of Snoopy's features:
* easily fetch the contents of a web page
* easily fetch the text from a web page (strip html tags)
* easily fetch the the links from a web page
* supports proxy hosts
* supports basic user/pass authentication
* supports setting user_agent, referer, cookies and header content
* supports browser redirects, and controlled depth of redirects
* expands fetched links to fully qualified URLs (default)
* easily submit form data and retrieve the results
* supports following html frames (added v0.92)
* supports passing cookies on redirects (added v0.92)
* 轻松抓取网页内容
* 轻松从网页获取文本（过滤 html标签）
* 轻松从网页获取链接
* 支持代理主机
* 支持基本 用户名/密码 认证
* 支持设置user_agent，refer，Cookie和header 内容
* 支持浏览器重定向和受控重定向深度
* 将指向完全授权网址的抓取链接展开（默认）
* 轻松提交表单数据和取回结果
* 支持以下html框架（v0.92添加）
* 支持在重定向上传递Cookie（v0.92添加）
REQUIREMENTS(依赖):
Snoopy requires PHP with PCRE (Perl Compatible Regular Expressions),
and the OpenSSL extension for fetching HTTPS requests.
史努比抓取HTTPS请求时需要安装PHP的PCRE和OpenSSL扩展。
CLASS METHODS(类方法):
fetch($URI)
-----------
This is the method used for fetching the contents of a web page.
$URI is the fully qualified URL of the page to fetch.
The results of the fetch are stored in $this-&results.
If you are fetching frames, then $this-&results
contains each frame fetched in an array.
这是用于获取网页内容的方法。
$URI是个完全授权的URL的网页。
获取的结果存储在$this-&results中。
如果您正在获取frames，那么$this-&results
的结果数组中包含每个frame提取的内容。
fetchtext($URI)
---------------
This behaves exactly like fetch() except that it only returns
the text from the page, stripping out html tags and other
irrelevant data.
这方法很像fetch()方法，但是它只返回
从网页的文本，剥离出了HTML标签等
不相关的数据。
fetchform($URI)
---------------
This behaves exactly like fetch() except that it only returns
the form elements from the page, stripping out html tags and other
irrelevant data.
这个方法跟fetch()也很像，但是它只返回符合条件的form元素。
fetchlinks($URI)
----------------
This behaves exactly like fetch() except that it only returns
the links from the page. By default, relative links are
converted to their fully qualified URL form.
这个方法跟fetch()也很像，但是它只返回符合条件的超链接。
默认情况下，链接会被转换为正常的网址形式。
submit($URI,$formvars)
----------------------
This submits a form to the specified $URI. $formvars is an
array of the form variables to pass.
此方法会向指定url提交form表单，$formvars是一个要传递变量的数组。
submittext($URI,$formvars)
--------------------------
This behaves exactly like submit() except that it only returns
the text from the page, stripping out html tags and other
irrelevant data.
类似与submit()方法，但是只返回文本内容。
submitlinks($URI)
----------------
This behaves exactly like submit() except that it only returns
the links from the page. By default, relative links are
converted to their fully qualified URL form.
类似与submit()方法，但是只返回链接。
CLASS VARIABLES:
(default value in parenthesis)
the host to connect to
the port to connect to
$proxy_host
the proxy host to use, if any
$proxy_port
the proxy port to use, if any
proxy can only be used for http URLs, but not https
the user agent to masqerade as (Snoopy v0.1)
referer information to pass, if any
cookies to pass if any
$rawheaders
other header info to pass, if any
$maxredirs
maximum redirects to allow. 0=none allowed. (5)
$offsiteok
whether or not to allow redirects off-site. (true)
$expandlinks
whether or not to expand links to fully qualified URLs (true)
authentication username, if any
authentication password, if any
http accept types (image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*)
where errors are sent, if any
$response_code responde code returned from server
headers returned from server
$maxlength
max return data length
$read_timeout
timeout on read operations (requires PHP 4 Beta 4+)
set to 0 to disallow timeouts
$timed_out
true if a read operation timed out (requires PHP 4 Beta 4+)
$maxframes
number of frames we will follow
http status of fetch
temp directory that the webserver can write to. (/tmp)
$curl_path
system path to cURL binary, set to false if none
(this variable is ignored as of Snoopy v1.2.6)
name of a file with CA certificate(s)
name of a correctly hashed directory with CA certificate(s)
if either $cafile or $capath is set, SSL certificate
verification is enabled
EXAMPLES(案例):
fetch a web page and display the return headers and
the contents of the page (html-escaped):
include "Snoopy.class.php";
$snoopy = new S
$snoopy-&user = "joe";
$snoopy-&pass = "bloe";
if($snoopy-&fetch("http://www.slashdot.org/"))
echo "response code: ".$snoopy-&response_code."&br&\n";
while(list($key,$val) = each($snoopy-&headers))
echo $key.": ".$val."&br&\n";
echo "&p&\n";
echo "&PRE&".htmlspecialchars($snoopy-&results)."&/PRE&\n";
echo "error fetching document: ".$snoopy-&error."\n";
submit a form and print out the result headers
and html-escaped page:
include "Snoopy.class.php";
$snoopy = new S
$submit_url = "http://lnk.ispi.net/texis/scripts/msearch/netsearch.html";
$submit_vars["q"] = "amiga";
$submit_vars["submit"] = "Search!";
$submit_vars["searchhost"] = "Altavista";
if($snoopy-&submit($submit_url,$submit_vars))
while(list($key,$val) = each($snoopy-&headers))
echo $key.": ".$val."&br&\n";
echo "&p&\n";
echo "&PRE&".htmlspecialchars($snoopy-&results)."&/PRE&\n";
echo "error fetching document: ".$snoopy-&error."\n";
showing functionality of all the variables:
include "Snoopy.class.php";
$snoopy = new S
$snoopy-&proxy_host = "my.proxy.host";
$snoopy-&proxy_port = "8080";
$snoopy-&agent = "( MSIE 4.01; MSN 2.5; AOL 4.0; Windows 98)";
$snoopy-&referer = "/";
$snoopy-&cookies["SessionID"] = 489l;
$snoopy-&cookies["favoriteColor"] = "RED";
$snoopy-&rawheaders["Pragma"] = "no-cache";
$snoopy-&maxredirs = 2;
$snoopy-&offsiteok =
$snoopy-&expandlinks =
$snoopy-&user = "joe";
$snoopy-&pass = "bloe";
if($snoopy-&fetchtext(""))
while(list($key,$val) = each($snoopy-&headers))
echo $key.": ".$val."&br&\n";
echo "&p&\n";
echo "&PRE&".htmlspecialchars($snoopy-&results)."&/PRE&\n";
echo "error fetching document: ".$snoopy-&error."\n";
fetched framed content and display the results
include "Snoopy.class.php";
$snoopy = new S
$snoopy-&maxframes = 5;
if($snoopy-&fetch("http://www.ispi.net/"))
echo "&PRE&".htmlspecialchars($snoopy-&results[0])."&/PRE&\n";
echo "&PRE&".htmlspecialchars($snoopy-&results[1])."&/PRE&\n";
echo "&PRE&".htmlspecialchars($snoopy-&results[2])."&/PRE&\n";
echo "error fetching document: ".$snoopy-&error."\n";
COPYRIGHT(版权):
Copyright(c)
ispi. All rights reserved.
This software is released under the GNU General Public License.
Please read the disclaimer at the top of the Snoopy.class.php file.
THANKS(鸣谢):
Special Thanks to:
Peter Sorger
help fixing a redirect bug
Andrei Zmievski
implementing time out functionality
Patric Sandelin
help with fetchform debugging
misc bug fixes with frames
&&相关文章推荐
参考知识库
* 以上用户言论只代表其个人观点，不代表CSDN网站的观点或立场
访问：8964次
排名：千里之外
原创：12篇
转载：28篇
(3)(5)(16)(8)(8)找好工作，快人一步代码人生技术博客
一行行枯燥的代码,却描绘人生的点点滴滴
倘若人生是一项编程，我愿用枯燥的代码，谱写人生的点点滴滴;
倘若人生是一套工序，我愿用笨拙的双手，刻录人生的朝朝暮暮;
倘若人生只是一场戏，我愿用诚实的面孔，演奏人生的始始终终.
当前位置：&
php curl模拟答题，总是400错误我想模拟这个网站的答题：/kf/html/index.html
他是在上面的页面登陆，之后到新的页面去答题，答题全部结束且真确之后，允许提交。
他是ajax提交，全部js验证通过后，回想后台发送ajax请求，完成页面。
他发送的数据倒是不复杂，但是在模拟他数据发送的时候，却总是出错，下面贴出代码和错误提示，望各位大神给予解救。
curl请求部分：
function&vget($url,$header&=&null){//模拟获取内容函数&&
&&&&$curl&=&curl_init();//启动一个CURL会话&&&&
&&&&curl_setopt($curl,&CURLOPT_URL,&$url);//要访问的地址&&&&&&&&&&&&&&&&
curl_setopt($curl,&CURLOPT_HTTPHEADER,&$browsers);//设置HTTP头
&&&&curl_setopt($curl,&CURLOPT_SSL_VERIFYPEER,&0);//&对认证证书来源的检查&&
&&&&curl_setopt($curl,&CURLOPT_SSL_VERIFYHOST,&1);//&从证书中检查SSL加密算法是否存在&&&&
&&&&curl_setopt($curl,&CURLOPT_USERAGENT,&$_SERVER['HTTP_USER_AGENT']);//&模拟用户使用的浏览器
curl_setopt($curl,CURLOPT_REFERER,'');&&&&
&&&&curl_setopt($curl,&CURLOPT_FOLLOWLOCATION,&1);//&使用自动跳转&&&&
&&&&curl_setopt($curl,&CURLOPT_AUTOREFERER,&1);//&自动设置Referer&&&&
&&&&curl_setopt($curl,&CURLOPT_HTTPGET,&1);//&发送一个常规的GET请求
curl_setopt($curl,&CURLOPT_COOKIEJAR,&$GLOBALS['cookie_file']);//保存cookie文件
if(isset($header)&&&&!is_null($header)){
curl_setopt($curl,&CURLOPT_HTTPHEADER,&$header);//设置头信息的地方&&
curl_setopt($curl,&CURLOPT_HEADER,&true);
&&&&curl_setopt($curl,&CURLOPT_TIMEOUT,&30);//&设置超时限制防止死循环&&&&
&&&&curl_setopt($curl,&CURLOPT_HEADER,&0);//&显示返回的Header区域内容&&&&
&&&&curl_setopt($curl,&CURLOPT_RETURNTRANSFER,&1);//&获取的信息以文件流的形式返回&&&&
&&&&$tmpInfo&=&curl_exec($curl);//&执行操作&&&&
&&&&if&(curl_errno($curl)){
&&&&&&&echo&'Errno'.curl_error($curl);
&&&&curl_close($curl);//&关闭CURL会话
&&&&return&$tmpI//&返回数据&&&&
function&vpost($url,$data){&//&模拟提交数据函数&
//$header&=&"Content-type:&application/x-www-form-urlencoded";//定义content-type为xml&&&
&&&&$curl&=&curl_init();&//&启动一个CURL会话&&&&
&&&&curl_setopt($curl,&CURLOPT_URL,&$url);&//&要访问的地址&&&&&&&&&&&&&&&&
&&&&curl_setopt($curl,&CURLOPT_SSL_VERIFYPEER,&0);&//&对认证证书来源的检查&&
curl_setopt($curl,&CURLOPT_HTTPHEADER,&$browsers);//设置HTTP头
&&&&curl_setopt($curl,&CURLOPT_SSL_VERIFYHOST,&1);&//&从证书中检查SSL加密算法是否存在&&&&
&&&&curl_setopt($curl,&CURLOPT_USERAGENT,&$_SERVER['HTTP_USER_AGENT']);&//&模拟用户使用的浏览器&&&&
&&&&curl_setopt($curl,&CURLOPT_FOLLOWLOCATION,&1);&//&使用自动跳转&&&&
curl_setopt($curl,&CURLOPT_REFERER,&'/kf/html/question.html??10');
&&&&//curl_setopt($curl,&CURLOPT_AUTOREFERER,&1);&//&自动设置Referer&&&&
&&&&curl_setopt($curl,&CURLOPT_POST,&1);&//&发送一个常规的Post请求&&&&
&&&&curl_setopt($curl,&CURLOPT_POSTFIELDS,&$data);&//&Post提交的数据包&&&&
&&&&curl_setopt($curl,&CURLOPT_COOKIEFILE,&$GLOBALS['cookie_file']);&//&读取上面所储存的Cookie信息&&&&
&&&&curl_setopt($curl,&CURLOPT_TIMEOUT,&30);&//&设置超时限制防止死循环&&&&
&&&&curl_setopt($curl,&CURLOPT_HEADER,&0);&//&显示返回的Header区域内容&&&&
&&&&curl_setopt($curl,&CURLOPT_RETURNTRANSFER,&1);&//&获取的信息以文件流的形式返回&&&&
&&&&$tmpInfo&=&curl_exec($curl);&//&执行操作&&&&
&&&&if&(curl_errno($curl)){&&&&
&&&&&&&echo&'Errno'.curl_error($curl);&&&&
&&&&curl_close($curl);&//&关键CURL会话&&&&
&&&&return&$tmpI&//&返回数据&&&&
执行部分：
header('Content-type:text/charset=utf-8');
$phone&=&'';
$area&&=&'10';
$cookie_jar&=&tempnam('./','cookie');//指定保存sessionid的cookie文件位置
$data&=&array(
'tel'=&urlencode($phone),
'area'=&urlencode($area)
$GLOBALS['cookie_file']&=&$cookie_
require_once('./functions.php');
$url&=&'/kf/html/index.html';
//&vget($url);
一行行枯燥的代码,却描绘人生的点点滴滴
网名：代码人生
职业：自由工作者
籍贯：广东省 广州市 天河区
社会工程学
_HTML/CSS书籍下载
_PHP书籍下载
_JavaScript
_JavaScript书籍下载
_Ext书籍下载
_jQuery书籍下载
_HTML5书籍下载
_Ajax书籍下载
_跨浏览器开发
_高性能WEB开发
_Web Service
_XML/SOAP书籍下载
_MySQL书籍下载
_Oracle技术
_Oracle管理
_Oracle开发
_Oracle书籍下载
_Oracle面试
_Oracle Exception
_Sql Server
_Sql Server书籍下载
_DB2书籍下载
_高性能数据库开发
_其他数据库
_移动开发书籍下载
_Android书籍下载
_Iphone书籍下载
_Windows Mobile
_Symbian书籍下载
_BlackBerry
_BlackBerry书籍下载
_移动软件开发
_电信IT应用开发
_企业信息化
_SharePoint
_Dynamics CRM
_Java Web开发
_Struts书籍下载
_Spring书籍下载
_Hibernate
_Hibernate书籍下载
_IBatis/Mybatis
_IBatis/Mybatis书籍下载
_J2ME书籍下载
_Java书籍下载
_Java Exception
_应用服务器
_JBoss书籍下载
_WebSphere
_ColdFusion
_软件架构设计
_微创软件开发
_PowerDesigner
_C#书籍下载
_ASP.NET书籍下载
_Visual Studio
_WinRT Metro
_.NET Framework
_VB Dotnet
_VB Dotnet书籍下载
_VC书籍下载
_.NET分析设计
_.NET组件控件
_.NET新技术
_DotNet Exception
_Linux/Unix
_Linux/Unix书籍下载
_多媒体/流媒体开发
_多媒体设计
_交互式开发
_Flash书籍下载
_图形/图像
_图像工具使用
_Flex书籍下载
_Silverlight
_C++书籍下载
_C语言书籍下载
_C++ Builder
_Ruby/Rails
_Ruby/Rails书籍下载
_perl/python
_perl/python书籍下载
_其他开发语言
_信息/网络安全
_IBM云计算
_Google技术
_高性能WEB开发
_高性能数据库开发
_高性能计算
_多核软件开发
_数据结构与算法
_网络设计维护
_电脑整机及配件
_装机与升级
_外设及办公设备
_交换机/路由器
_Windows书籍下载
_Windows2000
_Windows xp
_Windows Phone
_Windows Phone书籍下载
_Windows Server
_Oracle认证考试
_软件水平考试
_计算机图书
_计算机英语curl模拟登陆会员系统，登陆成功后网站路径错误，页面显示不出来【php吧】_百度贴吧
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&签到排名：今日本吧第个签到，本吧因你更精彩，明天继续来努力！
本吧签到人数：0成为超级会员，使用一键签到本月漏签0次！成为超级会员，赠送8张补签卡连续签到：天&&累计签到：天超级会员单次开通12个月以上，赠送连续签到卡3张
关注：132,083贴子：
curl模拟登陆会员系统，登陆成功后网站路径错误，页面显示不出来收藏
curl模拟登陆会员系统，登陆成功后网站路径错误，页面显示不出来，求帮我看看！获取验证码代码：&?php$login_url=&&;$cookie_file=dirname(__FILE__).'/cookie.tmp';$curl=curl_init();curl_setopt($curl,CURLOPT_URL,$login_url);curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);curl_setopt($curl, CURLOPT_COOKIEJAR, $cookie_file);curl_exec($curl);curl_close($curl);$code_url=&&;$ch=curl_init();curl_setopt($ch,CURLOPT_URL,$code_url);curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file); curl_exec($ch);curl_close($ch);?&登陆网站代码：&?phpif(isset($_POST['code'])){$code=$_POST['code'];$cookie_file=dirname(__FILE__).'/cookie.tmp';$login_url=&&;$postfile=&username=YTCDL&password=094941&yanzheng=$code&;$curl=curl_init();curl_setopt($curl,CURLOPT_URL,$login_url);curl_setopt($curl, CURLOPT_HEADER, 0);curl_setopt($curl, CURLOPT_RETURNTRANSFER, 0);curl_setopt($curl, CURLOPT_POST, 1);curl_setopt($curl, CURLOPT_POSTFIELDS, $postfile);
curl_setopt($curl, CURLOPT_COOKIEFILE, $cookie_file);curl_exec($curl);curl_close($curl);}?&&form action=&?act=data& method=&post&&&input name=&code& type=&text&/&&img src=&code.php&&&input name=&sub& value=&提交& type=&submit&/&&/form&求帮我看看
登录百度帐号推荐应用
为兴趣而生，贴吧更懂你。或2015年8月 PHP大版内专家分月排行榜第三
匿名用户不能发表回复！|
每天回帖即可获得10分可用分！小技巧：
你还可以输入10000个字符
(Ctrl+Enter)
请遵守CSDN，不得违反国家法律法规。
转载文章请注明出自“CSDN（www.csdn.net）”。如是商业用途请联系原作者。}